HTML-scraping

-
Ever had a need to process pages from a website that does not support any form of structural system integration like Web Service, RSS, REST, etc.
The only information available is ill-formed HTML; not even XHTML!?I have always been using HTML Agility for the .NET platform to perform such HTML screen-scraping.
Recently, found a number of Java equivalent toolkit to do the same:
Found this site that collects various toolkits for this purpose here.

Comments

Carly Fiorina said…
Hi all,

Acquiring data displayed on screen by capturing the text manually with the copy command or via software. Web pages are constantly being screen scraped in order to save meaningful data for later use. Thanks for sharing it.....

Data Scraping Software
04/11/2011, 16:18
Post a Comment

Popular posts from this blog

Understanding ITIL Service Management the UML way…

-
Image
I originally had a great deal of difficult reconciling the differences amongst ITIL Change Management, Incident Management, and Problem Management. Having researched on the subjects and understanding them better now, I’ve tried to put the concepts together using a UML class diagram to capture my understanding and to keep it imprinted. Similarly, I wanted to differentiate the various categories of changes (Standard, Emergency, Normal) using an easy to understand & self-explanatory manner – a UML Activity Diagram!
Read more

Apache Web Server Troubleshooting

-
How to troubleshoot Apache Web Server? A few standard commands to know by hard: To confirm the modules that are or will be loaded in Apache: apachectl –M [-f <conf filename>] To check the virtual host load pattern: apachectl –S [-f <conf filename>] To test a httpd.conf file: apachectl –t [-f <conf filename>] To maintain the http server: apachectl start|stop|restart To start with a specified configuration file: apachectl –f <conf filename> Create a minimalist httpd.conf. Something like: Listen 80 ServerName myserver.com:80 ServerAdmin admin@myserver.com #ServerRoot " . " ServerRoot " /usr/local/apache2 " User nobody Group nobody #DocumentRoot " ..\wwwroot " DocumentRoot " /home/data/html " ErrorLog " logs/basic_apache.log " LogLevel debug #LoadModule log_config_module modules/mod_log_config.dll LoadModule log_config_module modules/mod_log_config.so <IfModule log_config_module> LogFormat " %h %l
Read more