Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions lib/docs/filters/odin/clean_html.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module Docs
class Odin
class CleanHtmlFilter < Filter
def call
@doc = at_css('#pkg') || doc

css('nav').remove
doc
end
end
end
end
24 changes: 24 additions & 0 deletions lib/docs/filters/odin/entries.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
module Docs
class Odin
class EntriesFilter < Docs::EntriesFilter
def get_name
title = context[:html_title].gsub(/ \| Odin Programming Language/, "")
title
end

def get_type
if subpath.start_with?('docs')
"Documentation"
elsif subpath.start_with?('spec')
"Specifications"
end
end

def additional_entries
entries = []
entries
end
end
end
end

19 changes: 19 additions & 0 deletions lib/docs/filters/odin_packages/clean_html.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
module Docs
class OdinPackages
class CleanHtmlFilter < Filter
def call
@doc = at_css('#pkg') || doc

css('.pkg-breadcrumb').remove
css('.a-hidden').remove
css('.doc-source').remove
css('.odin-search-wrapper').remove
css('#pkg-sidebar').remove
css('#odin-search-info').remove
css('#odin-search-results').remove

doc
end
end
end
end
40 changes: 40 additions & 0 deletions lib/docs/filters/odin_packages/entries.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
module Docs
class OdinPackages
class EntriesFilter < Docs::EntriesFilter
def get_name
breadcrumbs = css(".pkg-breadcrumb > ol > li")
if breadcrumbs
if breadcrumbs[1]
breadcrumbs[1].content
elsif breadcrumbs[0]
breadcrumbs[0].content
end
end
title = context[:html_title].gsub(/- pkg.odin-lang.org/, "")
title = title.gsub(/^package /, "")
title
end

def get_type
breadcrumb_base = css(".pkg-breadcrumb > ol > li")
doc_directory = css(".doc-directory")
if breadcrumb_base[0]
breadcrumb_base[0].content
elsif doc_directory
title = context[:html_title].gsub(/ library - pkg.odin-lang.org/, "")
title
elsif context[:html_title].starts_with?('package')
'Packages'
else
'Docs'
end
end

def additional_entries
entries = []
entries
end
end
end
end

38 changes: 38 additions & 0 deletions lib/docs/scrapers/odin.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
module Docs
class Odin < UrlScraper
self.name = 'Odin'
self.slug = 'odin'
self.type = 'odin'
self.release = 'latest'
self.base_url = 'https://odin-lang.org/'
self.root_path = 'https://odin-lang.org/'
self.initial_paths = %w(docs spec)

self.links = {
home: 'https://odin-lang.org/',
code: 'https://github.com/odin-lang/Odin'
}

html_filters.push 'odin/entries', 'odin/clean_html'
options[:download_images] = false

options[:container] = '.odin-main'

options[:only_patterns] = [/docs/, /spec/]
options[:trailing_slash] = false

options[:skip] = %w(
docs/examples
docs/nightly
docs/odin-book
docs/spec
docs/packages
)

options[:attribution] = <<-HTML
&copy; 2016-#{Date.today.year} Ginger Bill<br>
Licensed under the 3-clause BSD License.
HTML
Comment on lines +32 to +35

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The license has been changed, see odin-lang/Odin@842cfee.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it! I'll get this updated!


end
end
28 changes: 28 additions & 0 deletions lib/docs/scrapers/odin_packages.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
module Docs
class OdinPackages < UrlScraper
self.name = 'Odin Packages'
self.slug = 'odin_packages'
self.type = 'odin_packages'
Comment on lines +3 to +5

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to have this separate scraper to be merged with the main odin. Using one without the other does not seem overly helpful to me. Also, this follows practice we've been following for other scrapers (such as crystal or rust).

@jask06 jask06 Jun 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This makes a lot of sense. When I was looking, I didn't recognize there were parsers with multiple URLs that could be executed in the same scrape (my own skill issue). Thank you for pointing that out!

I'll look at those and get that updated!

self.release = 'latest'
self.base_url = 'https://pkg.odin-lang.org/'
self.initial_paths = %w(base core vendor)

options[:trailing_slash] = false

self.links = {
home: 'https://odin-lang.org/',
code: 'https://github.com/odin-lang/Odin'
}

html_filters.push 'odin_packages/entries', 'odin_packages/clean_html'

options[:download_images] = false
options[:container] = '.odin-main'

options[:attribution] = <<-HTML
&copy; 2016-#{Date.today.year} Ginger Bill<br>
Licensed under the 3-clause BSD License.
HTML

end
end
Binary file added public/icons/docs/odin/16.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/icons/docs/odin/16@2x.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions public/icons/docs/odin/SOURCE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://github.com/odin-lang/odin-lang.org/blob/master/static/favicon.svg
Binary file added public/icons/docs/odin_packages/16.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/icons/docs/odin_packages/16@2x.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions public/icons/docs/odin_packages/SOURCE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://github.com/odin-lang/odin-lang.org/blob/master/static/favicon.svg