
|
About Kolinka: At Project Kolinka we are developing a new way to search community driven web forums and message boards. Our goal is to build a complete index of these forums and provide an easy tool to search across millions of messages. We hope to increase traffic and visibilty to web forums which are an overlooked area of the web by most search engines. Forum Spider Info: The Kolinka spider only crawls forum content, requesting pages at a moderate rate of 1 page per second. In comparison Google and other search engines can request up to 10 pages per second. Our spider should not degrade system performance and only opens one connection to your website. The Kolinka spider follows most polices defined in your robots.txt file. To conserve your bandwidth the spider attempts to visit the printable version of each post. However, the final indexed link will point to the regular post page (so your ads, navigation, etc. are still seen). If you are currently blocking other bots and would like to allow Kolinka to index your forums please add the following lines to your robots.txt file: User-agent: Project Kolinka Forum Search (www.kolinka.com) Disallow: For more information on how to control spider behavior visit: http://www.robotstxt.org/wc/exclusion.html Contact Us: This project is still in development, if you would like to suggest a forum or have questions about our spider please email us at: questions@kolinka.com. If for any reason you would like us to stop crawling your forum please contact us and we will manually remove it. |