ExcavatorSharp is a multi-threaded server for scraping web data. It converts HTML code into a structured array of object data. The library allows scraping data from multiple sites simultaneously, in parallel, within a single running application. Create scraping tasks and perform data extraction on a schedule.
The library can be used separately as crawler or parser. Work with sitemap, robots.txt is supported. It supports working with gzip / deflate compression. Work with links by masks, automatic link re-indexing and so on.
Attention! Only x64 versions are supported for .NET 4.5.2 and 4.6 platforms. AnyCPU build does not support! You will NOT be able to run the library when building AnyCPU. This is caused by the features of CEF.
See the version list below for details.
Install-Package ExcavatorSharp.WebScraper.x64 -Version 1.0.4
dotnet add package ExcavatorSharp.WebScraper.x64 --version 1.0.4
<PackageReference Include="ExcavatorSharp.WebScraper.x64" Version="1.0.4" />
paket add ExcavatorSharp.WebScraper.x64 --version 1.0.4
Changes in the Crawler properties: properties validation, URLs respecting fixes. Updating linked libraries.
This package is not used by any popular GitHub repositories.