dolospy

Name: dolospy
Published: 2026-01-22
License: https://mit-license.org/

Open this dataset in the live repository

Important usage condition: Use and redistribution are governed by MIT. Review and comply with the license before using the data.

Persistent identifier: doi:10.60507/FK2/LYIDHI
Published version: 1.0
Publication date: 2026-01-22
License: MIT

Description

This repository contains a re-implementation of the rolling AST hash algorithm from Dolos (https://dolos.ugent.be/) in C++ with Python bindings. It is 8x faster for indexing and 10,000x faster in reporting compared to the original JavaScript implementation (cf. BENCHMARKING.md).

Creators

Swierzy, Ben

Keywords

code similarity, rolling ast hash

Files

Before downloading: Use and redistribution are governed by MIT. Review and comply with the license before using the data.

File	Type	Bytes	Checksum
readme.txt	text/plain	2706	MD5 ac7e464c7be5951689b2ef9e709fb0cb
BENCHMARKING.md	text/markdown	2003	MD5 1db4fa3b5d15063f6f3f0e2aab7aaa3d
tokenizer.h	text/x-chdr	245	MD5 b424721ba77516ece32504b0803d2d78
dolospy.pyi	text/x-python3	771	MD5 7565c2e54f0db99774ed9f57ae2dc7d3
CMakeLists.txt	text/plain	530	MD5 6a6ed6ad505aedddc40133bfcdaace1d
hashing.cpp	text/x-c++src	427	MD5 2abc7d2ce7037b22e912555e3351cd21
__main__.py	text/x-python	4032	MD5 2f3b777ad646f8ecdfffe9a4f6f15d2d
python.cpp	text/x-c++src	2434	MD5 fc9ea80f5e4295af554dde440b48e040
tokenizer.cpp	text/x-c++src	1802	MD5 7a601aca257e4a9a3a4d060d21491368
__init__.py	text/x-python	23	MD5 e530edf264988f6a4c4babd93c530c3d
interface.cpp	text/x-c++src	1175	MD5 3776958457801dec7fc20cf6ef4ab603
pyproject.toml	application/toml	289	MD5 4368885f11e8fa8a025a7038a9674d67
main.cpp	text/x-c++src	1371	MD5 ec5984e3784fd2070846100648737a14
index.h	text/x-chdr	1128	MD5 1bc1902c00ef14ba0dc1eda8e83ee433
hashing.h	text/x-chdr	2693	MD5 c75f67e5eb94720110d8f5bf4edabdbc
interface.h	text/x-chdr	222	MD5 f70d3f56dfecdcbe6a0c7eabcfa484ef
BUILD.md	text/markdown	203	MD5 673451ff60913a9e6ac59a6f085a76c5
index.cpp	text/x-c++src	4925	MD5 d9342731f06c2c9ac192ebf7a8e3e395

Citation

Swierzy, Ben, 2026-01-22, dolospy, doi:10.60507/FK2/LYIDHI, V1.0

Additional Dataverse fields

Id	559
Dataset Type	dataset
Internal Version Number	19
Latest Version Publishing State	RELEASED
Release Time	2026-01-22T11:47:21Z
Create Time	2026-01-14T11:57:38Z
Citation Date	2026-01-22
File Access Request	True

Export metadata

Static metadata exports available for this published dataset version:

Complete Dataverse metadata

Citation Metadata

Title title

dolospy

Subtitle subtitle

Efficient implementation of the core algorithm for the plagiarism detection tool "Dolos"

Alternative URL alternativeURL

https://github.com/BlobbyBob/AletheiaJSVersionDetection

Author author

Author Name
Swierzy, Ben
Author Affiliation
University of Bonn
Author Identifier Scheme
ORCID
Author Identifier
Type Name
authorIdentifier
Multiple
False
Type Class
primitive
Value
https://orcid.org/0009-0003-0485-4791
Expandedvalue
Person Name
Swierzy, Ben
@id
https://orcid.org/0009-0003-0485-4791
Scheme
ORCID
@type
https://schema.org/Person

Dataset Contact datasetContact

Dataset Contact Name
Swierzy, Ben
Dataset Contact Affiliation
University of Bonn
Dataset Contact Email
swierzy@cs.uni-bonn.de

Ds Description dsDescription

Ds Description Value
This repository contains a re-implementation of the rolling AST hash algorithm from Dolos (https://dolos.ugent.be/) in C++ with Python bindings. It is 8x faster for indexing and 10,000x faster in reporting compared to the original JavaScript implementation (cf. BENCHMARKING.md).

Subject subject

Computer and Information Science

Subject Refinement subjectRefinement

Subject Refinement Value
Computer Science: Software Engineering and Programming Languages [409-02]
Subject Refinement Value
Computer Science: Security and Dependability [409-03]

Free Keyword freeKeyword

Free Keyword Value
code similarity
Free Keyword Value
rolling ast hash

Publication publication

Publication Relation Type
IsSupplementTo
Publication Citation
Ben Swierzy, Marc Ohm, and Michael Meier. 2026. Insecure Ingredients? Exploring Dependency Update Patterns of Bundled JavaScript Packages on the Web. In 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3744916.3787780
Publication IDType
doi
Publication IDNumber
10.1145/3744916.3787780
Publication URL
https://dl.acm.org/doi/abs/10.1145/3744916.3787780
Publication Relation Type
IsSupplementTo
Publication Citation
Swierzy, Ben, Marc Ohm, and Michael Meier. "Insecure Ingredients? Exploring Dependency Update Patterns of Bundled JavaScript Packages on the Web." arXiv preprint arXiv:2512.15447 (2025).
Publication IDType
doi
Publication IDNumber
10.48550/arXiv.2512.15447
Publication URL
https://arxiv.org/abs/2512.15447

Language language

English

Depositor depositor

Swierzy, Ben

Date Of Deposit dateOfDeposit

2026-01-14

Kind Of Data kindOfData

program source code

Related Material relatedMaterial

GitHub Repository: https://github.com/BlobbyBob/AletheiaJSVersionDetection
Evaluation Scripts that utilize dolospy: https://doi.org/10.60507/FK2/F4VQRH

Related Datasets relatedDatasets

Internet Measurement Dataset: https://doi.org/10.60507/FK2/AHYGMN

Software Metadata (CodeMeta v2.0)

Code Version codeVersion

0.3.5

Development Status developmentStatus

Concept

Code Repository codeRepository

https://github.com/BlobbyBob/AletheiaJSVersionDetection

Application Category applicationCategory

Analysis

Application Sub Category applicationSubCategory

Static Code Analysis

Programming Language programmingLanguage

C++
Python

Runtime Platform runtimePlatform

Python 3.12

Build Instructions buildInstructions

https://github.com/BlobbyBob/AletheiaJSVersionDetection/tree/main/dolospy/BUILD.md

Software Requirements Item softwareRequirementsItem

Software Requirements
pybind11 >= 3
Software Requirements Info Url
https://pybind11.readthedocs.io/en/stable/index.html
Software Requirements
tree-sitter >= 0.25.10
Software Requirements Info Url
https://tree-sitter.github.io/tree-sitter/
Software Requirements
tree-sitter-javascript >= 0.25.0
Software Requirements Info Url
https://github.com/tree-sitter/tree-sitter-javascript
Software Requirements
boost-json >= 1.81.0.2
Software Requirements Info Url
https://www.boost.org/

Processor Requirements processorRequirements

Readme readme

https://github.com/BlobbyBob/AletheiaJSVersionDetection/blob/main/README.md

Expected crawler behaviour

Use a stable, truthful User-Agent with product/version and a working contact URL. Across all IP addresses and HTTP connections used by one crawler identity, allow no more than 5 requests in flight and wait at least 20 seconds between request starts. Crawl URLs listed in the catalog sitemap, including file pages and download URLs when they are published, use conditional requests, honor Retry-After, and apply exponential backoff after errors.

The welcome page may link to the interactive repository for human navigation. Automated clients must not treat that human link as a catalog crawl target.

Read the live machine-readable crawler policy before and during a crawl. Stop crawling when it reports CPU or memory utilization at or above 80% and 80% respectively.