This is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...). It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to... More information

Simple web crawler that return IObservable using Reactive Extension(Rx) and async await.

Crawl and index your (static) asp.net website for searching using an MSBuild target. Simple search query interface. Uses Lucene.Net and NCrawler.

Library allows you to extract different data from text or web sources, analyze text content, extract different information and write your own data mining & extracting applications and services.

FSharp Web Crawler

A minimal multithreaded web crawler

This is my sample package

The crawling/indexing component of the Sitecore Search Contrib project

Crawler for main social networks

The searching/querying component of the Sitecore Search Contrib project