Contributing to Ariadne

Reporting bugs & requesting features

Development setup

Prerequisites

Java 11

JAVA_HOME=/usr/lib/jvm/java-11-openjdk or equivalent for the deployed Synapse Spark 3.5 line; use Java 21 for Fabric Spark 4.1.

Maven 3.x

Used for build, test, packaging.

scalafmt / scalafix / scalastyle

Enforced by the build. Run mvn scalafmt:format to format; scalafix (RemoveUnused + OrganizeImports) and scalastyle (adapted from Apache Spark) run in the verify phase, so mvn verify (and CI) fail on any violation.

Git hooks (recommended)

scalafix and scalastyle run in the verify phase, behind a full compile and test run, so a stray unused import is only reported after a complete CI cycle. The same checks take a few seconds locally. Install the pre-commit hook to run them before the commit instead:

dev/scripts/install-git-hooks.sh

This points core.hooksPath at the version-controlled dev/hooks directory, so the hooks stay in sync with the repository rather than being copied into .git/hooks and going stale. Use --check to see the current state and --uninstall to remove it.

The pre-commit hook exits immediately unless the commit stages Scala sources. When it does, it runs scalafmt:format and re-stages what it reformatted, then compiles and runs scalafix and scalastyle. Files with both staged and unstaged changes are never re-staged automatically — the hook reports them and stops, because git add would sweep in work you deliberately left out of the commit. scalafmt formats every source directory configured in pom.xml rather than only the staged files, so if it rewrites a Scala file that is not part of your commit the hook names that file and stops, leaving you to stage or revert it.

git commit --no-verify

Skip the hook for one commit.

ARIADNE_SKIP_HOOKS=1

Disable the hook entirely.

ARIADNE_HOOK_SKIP_SCALAFIX=1

Skip only the compile-dependent scalafix step, which is the slowest part.

Build & test

# Run all tests
mvn test

# Run a single test suite
mvn test -Dsuites="dev.cjfravel.ariadne.IndexTests"

# Run one test within a suite (space separator; @ means exact-name match)
mvn test -Dsuites="dev.cjfravel.ariadne.IndexTests @my test name"

# Drop the @ to match any test whose name contains the string
mvn test -Dsuites="dev.cjfravel.ariadne.IndexTests my test"

# Package (produces the shaded JAR)
mvn package

The test phase also runs dev/scripts/readme-has-version.sh, which fails if the version in pom.xml isn't reflected in the docs. Always update both when bumping the version.

Code style

• Format with scalafmt (mvn scalafmt:format). Imports are organized and unused code removed by scalafix (RemoveUnused + OrganizeImports), and scalastyle (adapted from Apache Spark) enforces static checks. All three run in the build and CI, so a violation fails the build. Install the pre-commit hook to catch violations before they reach CI.
• Standard Scala naming: camelCase for methods/vals, PascalCase for types.
• Expression-oriented style — prefer if/else expressions over return.
• Use import scala.collection.JavaConverters._ (the project is Scala 2.12; CollectionConverters is 2.13+).
• Every public class, trait, object, and method must have Scaladoc.
• logger.warn(...) for normal operational messages — this is intentional and matches how Spark surfaces logs in cluster environments. Reserve debug for verbose/low-value entries.
• No silent catch blocks. Always log the exception before swallowing or rethrowing.

Backwards compatibility

Ariadne aims to be backwards compatible between releases — existing indexes should keep working when users upgrade, and the public Scala API shouldn't break under feet. Treat that as the default expectation when proposing changes.

That said, Ariadne is still in beta. If a breaking change clearly benefits the project more than the disruption costs, it can land — but only with the maintainer's explicit sign-off. Don't assume a breaking change will be accepted just because it's cleaner; open an issue or discussion to align first.

Shading & relocation

Ariadne bundles Guava and Gson, relocated under dev.cjfravel.ariadne.shaded.*, so they can't collide with whatever versions the host Spark runtime ships. Relocation is easy to break in two different directions, and both have caused real bugs here.

Adding a bundled dependency

A new relocated dependency needs three coordinated edits, not one: a <relocation> entry in the shade plugin config, a matching <artifactSet><include> entry, and a reject_entry assertion in dev/scripts/package-contents-tests.sh confirming the original package is absent from the packaged jar. Miss one and the package ships unrelocated and collides at runtime — which is exactly how com.google.thirdparty, error_prone_annotations and j2objc-annotations each had to be retrofitted after the fact.

Staying safe when consumers relocate Ariadne

Downstream builds sometimes shade Ariadne itself into their own namespace. The Maven Shade Plugin rewrites bytecode references but leaves the Scala ScalaSignature pickle byte-for-byte identical, so a relocated class loads under its new package while its pickle still describes the old one. Any type resolved through Scala reflection then can't be found, usually surfacing as ENCODER_NOT_FOUND during an index update.

• Never use an Ariadne-defined case class as a Spark Dataset or encoder element type. Use a DataFrame with an explicit StructType, or build the encoder explicitly with Encoders.tuple, Encoders.STRING and friends.
• Prefer the two-argument udaf(aggregator, inputEncoder) overload — the single-argument one derives the input encoder reflectively.
• .as[T], .toDS() and emptyDataset[T] are safe only for standard-library types such as String and tuples of primitives, which relocation never rewrites.
• Encoders.javaSerialization[T] and Encoders.kryo[T] are safe — they take a ClassTag, which compiles to a classOf reference that shade does rewrite. Only TypeTag-based derivation is unsafe.
• Never hardcode dev.cjfravel.ariadne in a string used for lookup (Class.forName, Spark extension or config keys, service-loader entries). Relocation rewrites the class, not the string. Package names inside Scaladoc examples are fine.

To verify a change by hand, package target/classes into a jar, install it locally, then shade it in a scratch project with a <relocation> onto a different prefix and exercise the affected code path.

Submitting a pull request

1. Fork the repository and branch from main.
2. Make your changes, ensuring mvn test passes.
3. Update documentation if your changes affect the public API.
4. If you changed any public class, trait, method, or scaladoc, no extra step is needed — the API reference is generated and published automatically from main. Preview it locally with dev/scripts/build-docs-site.sh.
5. Don't change the version in pom.xml or the docs — versioning is managed by the maintainer.
6. Open a pull request with a clear description.

AI-assisted contributions

AI assistance (Copilot, ChatGPT, etc.) is welcome — but every PR must reflect genuine human understanding:

• Review and comprehend all AI-generated code before submitting. If you can't explain what it does and why, it's not ready.
• Purely AI-generated PRs with no human review will be closed. We need contributors who can discuss their changes, respond to feedback, and reason about edge cases.
• Attribute AI assistance honestly. There's no stigma — just be transparent.

The goal is quality contributions from people who understand what they're submitting, regardless of how they got there.

Questions

Open a GitHub Discussion.