Additionally, when the website uses a web form to query and return results, the web address is read by the web server or application server, the address is then parsed for parameters, and the parameters are passed to the database behind the website which control the results returned.
There are techniques that utilize these facts to extract large amounts of data from the backend database behind a website through a series of crafted web page requests.
Collectively these techniques are called Web Scraping.
Two of the key techniques of web scraping are URL Hacking and HTML parsing.
Koob, P., Ivliyeva, Russian National Corpus Web Scraping Project 2019-2020. Missouri S&T, IT and ALP departments. [Electronic resource].
Arts, Languages, and Philosophy
© 2020 The Authors, All rights reserved.
01 Feb 2020