Spidits is a compliance-first AI and startup intelligence aggregator. We only crawl publicly exposed metadata (headlines, canonical links, and publication dates) from verified directories and blogs. We strictly enforce a no-caching and metadata-only policy—we never store or republish article body copy or images.
If you are a publisher and wish to have your content removed or opt out of indexing completely, please submit a request. You do not need to prove a legal copyright violation; we respect all publisher preferences.
To request content removal:
Spidits programmatically respects all standard search indexing and crawling configurations. If you want to configure technical boundaries without removing your feed entirely, you can use the following:
User-agent: Spidits followed by Disallow: / in your robots.txt file. Our ingestion pipeline reads and enforces robots.txt rules dynamically.X-Robots-Tag with noindex, Spidits will automatically flag your articles for audit review.