What web technology should I use? I am confused.

[ekbass]

I want to do the followings when someone visits my web site.

1. Access to one internet forum web site (such as this one).
2. Read contents of each thread and summarize them.
3. Display the results to the visitor's browser.

To perform summarization, I would probably need some tools such as XML parser, some datatypes (eg. Set, Dictionary, List, etc.) and natural language processing tools that are comparable to the NLTK in Python.

I know that I would need Web technology that provides rich libraries, but seems like there are many of them (Ruby on Rails, JQuery, Dojo, JavaScript, etc.) and I do not know which one would be the best for my purpose. What web technology should I use and how should I use it? Btw, I should do this with client-side scripting, correct?

 

[office politics]

i'd suggest you build a web scraper to collect information from message boards. This is assuming you do not have db access to the forum. This loads the raw data into a database.

Then, you would want to create a summarize process. This would be handled by the DBMS. I would try to use Query language to complete this.

Then, use a server side web language to show the output. Optionally, you could use the query language to show the results if the user can have db access.

 

[kmote]

i'd suggest you build a web scraper to collect information from message boards. This is assuming you do not have db access to the forum. This loads the raw data into a database.

Then, you would want to create a summarize process. This would be handled by the DBMS. I would try to use Query language to complete this.

Then, use a server side web language to show the output. Optionally, you could use the query language to show the results if the user can have db access.

... except of course, by collecting the data in advance it's going to be stale by the time the user accesses it.

I want to do the followings when someone visits my web site.

1. Access to one internet forum web site (such as this one).
2. Read contents of each thread and summarize them.
3. Display the results to the visitor's browser.

To perform summarization, I would probably need some tools such as XML parser, some datatypes (eg. Set, Dictionary, List, etc.) and natural language processing tools that are comparable to the NLTK in Python.

I know that I would need Web technology that provides rich libraries, but seems like there are many of them (Ruby on Rails, JQuery, Dojo, JavaScript, etc.) and I do not know which one would be the best for my purpose. What web technology should I use and how should I use it? Btw, I should do this with client-side scripting, correct?

Client-side is definitely preferable IMO, though I would look into doing this through a browser plugin.

 

[office politics]

... except of course, by collecting the data in advance it's going to be stale by the time the user accesses it.

message board records are fairly static though.