apache/spark: Maven, PySpark and SparkR in one config

Spark ships a Scala engine, Java extension points, three PySpark distributions and an R package from one Maven build. The config checks each part's layout and manifest values, and found 4 Java files named against the PascalCase convention the rest follow. This config was written to test alint; Spark doesn't use alint.

What alint found

.alint.yml

extends:
  - alint://bundled/java@v1

⚠︎ warning java-sources-pascal-case

Four Java files aren't named in PascalCase: three named after Murmur3_x86_32, and typed.java.

The bundled GitHub Actions rules (alint 0.16.1, on the workflows as of 7 May 2026) flag 122 action references not pinned to a commit, and 9 of the 71 workflows that declare no permissions, among them maven_test.yml and benchmark.yml.

The config

Spark's four languages sit at different layers: Scala is the engine, Java adds extension points, PySpark ships as three PyPI distributions and SparkR wraps the engine for R. The build is Maven. The config checks a pom.xml in every module, the root POM's values as XML, the PySpark variants and SparkR's DESCRIPTION and NAMESPACE. It extends seven bundled rulesets, among them apache/governance and java, and runs the lint scripts from command rules.

.alint.yml

- id: spark-root-pom-apache-parent
  kind: xml_path_equals
  paths: pom.xml
  path: "$.project.parent.artifactId"
  equals: "apache"
  level: error
  policy_url: "https://maven.apache.org/pom/asf/"

xml_path_equals reads pom.xml as XML, so a commented-out or nested <parent> can't satisfy it. Four more rules check the root POM's group, artifact ID and Java version the same way.

.alint.yml

- id: spark-python-packaging-variant-shape
  kind: for_each_dir
  select: "python/packaging/{classic,client,connect}"
  require:
    - kind: file_exists
      paths: "{path}/setup.py"
    - kind: file_exists
      paths: "{path}/setup.cfg"
  level: error

PySpark ships as three distributions. Each variant directory needs its setup.py and setup.cfg, and three more rules check each one's package name.

.alint.yml

- id: spark-scala-no-log4j-direct
  kind: import_gate
  paths: "**/*.scala"
  language: generic
  import_pattern: '^\s*import\s+(\S+)'
  forbid: '^org\.apache\.log4j'
  level: error

scalastyle's ban on Log4j 1, as an import_gate rule: Spark logs through Log4j 2, and nothing may import org.apache.log4j. import_gate has no Scala preset; import_pattern gives it the syntax.

Speed

In the May run, on a checkout of 28,917 files, the rules that don't call other tools took 1.35 s.

What stays outside alint

What needs a compiler or a release stays with Spark's tools:

MiMa, which checks binary compatibility between Spark versions;
the checks that parse Scala, Python or protobuf code;
about 14 operational scripts in dev/ and the release scripts;
the per-language linters themselves, which the config runs from command rules.