squirm

Version, currently 0.2.03 versions

github.com/squirm-framework/squirm

This was the night of the crawling terror!

32 stars
0 dependents
License: MIT

Installation

# Add this to your shard.yml
dependencies:
  squirm:
    github: squirm-framework/squirm
    version: ~> 0.2.0

Then run:

shards install

shard.yml

Crystal
~> 1.2.0
License
MIT
Author
Giorgi Kavrelishvili <giorgi.kavrelishvili@pm.me>

Dependencies

Runtime Dependencies

  • crest*github: mamantoha/crest
  • redis*github: stefanwille/crystal-redis
  • lexbor*github: kostya/lexbor
  • robots*github: grkek/robots.cr, branch: master
  • rocksdb1.1.0github: maiha/rocksdb.cr
  • marionette*github: watzon/marionette, branch: master

Development Dependencies

  • ameba*github: crystal-ameba/amebadev

README

<p align="center" width="100%">
    <img src="https://github.com/squirm-framework/medias/blob/master/framework.svg" height="250" href="https://github.com/squirm-framework/squirm">
</p>

<p align="center">
    This was the night of the <ins>crawling terror</ins>!<br><br>
</p>

Squirm is a framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.

## Installation

1. Add the dependency to your `shard.yml`:

   ```yaml
   dependencies:
     squirm:
       github: squirm-framework/squirm
   ```

2. Run `shards install`

## Usage

View the usage in the examples directory.

## Contributing

1. Fork it (<https://github.com/squirm-framework/squirm/fork>)
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request

## Contributors

- [Giorgi Kavrelishvili](https://github.com/grkek) - creator and maintainer