squirm
Version, currently 0.2.03 versions
- 0.2.0latestDec 22, 2023
- 0.1.1not indexedDec 22, 2023
- 0.1.0not indexedDec 22, 2023
github.com/squirm-framework/squirm
This was the night of the crawling terror!
32 stars
0 dependents
License: MIT
Installation
# Add this to your shard.yml
dependencies:
squirm:
github: squirm-framework/squirm
version: ~> 0.2.0Then run:
shards installshard.yml
- Crystal
~> 1.2.0- License
- MIT
- Author
- Giorgi Kavrelishvili
Dependencies
Runtime Dependencies
- crest*github: mamantoha/crest
- redis*github: stefanwille/crystal-redis
- lexbor*github: kostya/lexbor
- robots*github: grkek/robots.cr, branch: master
- rocksdb1.1.0github: maiha/rocksdb.cr
- marionette*github: watzon/marionette, branch: master
Development Dependencies
- ameba*github: crystal-ameba/amebadev
README
Squirm is a framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.
Installation
-
Add the dependency to your
shard.yml:dependencies: squirm: github: squirm-framework/squirm -
Run
shards install
Usage
View the usage in the examples directory.
Contributing
- Fork it (https://github.com/squirm-framework/squirm/fork)
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create a new Pull Request
Contributors
- Giorgi Kavrelishvili - creator and maintainer
Documentation
Built from the current release. The first visit to a release nobody has asked for starts its build.
Links
This release
- Version
0.2.0- Tagged
- Dec 22, 2023
- Commit
c50a8b430a8b- Crystal
~> 1.2.0- Indexed
- yes
Dependents
No indexed shard depends on this one yet.
Repository
github.com/squirm-framework/squirm
Metadata
- Created
- Aug 12, 2026
- Updated
- Sep 24, 2026
- Synced
- Sep 24, 2026
- Versions
- 3