Shard Detail

lexbor v3.0.4

Fast HTML5 Parser with CSS selectors. This is successor of myhtml and expected to be faster and use less memory.

Install & Use

Add the following code to your project's shard.yml under:

dependencies to use in production
- OR -
development_dependencies to use in development 


lexbor:
  github: kostya/lexbor

Readme

Lexbor

Build Status

Fast HTML5 Parser with CSS selectors (based on new lexborisov's HTML5 parser lexbor). This is successor of myhtml and expected to be faster and use less memory. Usage is almost equal to myhtml.

Installation

Add this to your application's shard.yml:

dependencies:
  lexbor:
    github: kostya/lexbor

And run shards install (Installation require cmake, make sure it installed)

Usage example

require "lexbor"

html = <<-HTML
  <html>
    <body>
      <div id="t1" class="red">
        <a href="/#">O_o</a>
      </div>
      <div id="t2"></div>
    </body>
  </html>
HTML

lexbor = Lexbor::Parser.new(html)

lexbor.nodes(:div).each do |node|
  id = node["id"]?

  if first_link = node.scope.nodes(:a).first?
    href = first_link["href"]?
    link_text = first_link.inner_text

    puts "div with id #{id} have link [#{link_text}](#{href})"
  else
    puts "div with id #{id} have no links"
  end
end

# Output:
#   div with id t1 have link [O_o](/#)
#   div with id t2 have no links

Css selectors example

require "lexbor"

html = <<-HTML
  <html>
    <body>
      <table id="t1">
        <tr><td>Hello</td></tr>
      </table>
      <table id="t2">
        <tr><td>123</td><td>other</td></tr>
        <tr><td>foo</td><td>columns</td></tr>
        <tr><td>bar</td><td>are</td></tr>
        <tr><td>xyz</td><td>ignored</td></tr>
      </table>
    </body>
  </html>
HTML

lexbor = Lexbor::Parser.new(html)

p lexbor.css("#t2 tr td:first-child").map(&.inner_text).to_a
# => ["123", "foo", "bar", "xyz"]

p lexbor.css("#t2 tr td:first-child").map(&.to_html).to_a
# => ["<td>123</td>", "<td>foo</td>", "<td>bar</td>", "<td>xyz</td>"]

More Examples

examples

Development Setup:

git clone https://github.com/kostya/lexbor.git
cd lexbor
make
crystal spec

Benchmark

Parse google results page(600Kb) 1000 times, and 5000 times css select.

lexbor-program myhtml-program crystagiri-program nokogiri-program

Running on Ryzen 3800x.

Lang Lib Parse time, s Css time, s Memory, MiB
Crystal lexbor 2.48 0.48 8.8
Crystal myhtml(+modest) 3.13 0.77 11.7
Ruby 2.7 Nokogiri(libxml2) 9.44 54.70 148.7
Crystal Crystagiri(libxml2) 11.90 - 25.1

Running on Apple M1.

Lang Lib Parse time, s Css time, s Memory, MiB
Crystal lexbor 1.54 0.49 20.9
Crystal myhtml(+modest) 2.24 0.67 19.8
Ruby 2.7 Nokogiri(libxml2) 9.87 55.72 163.8
Crystal Crystagiri(libxml2) 11.15 - 17.4