Validate Avro Schema Compatibility in CI

TL;DR

Store the current released Avro schema as a baseline, parse both schemas with Apache Avro, and fail a Gradle test when SchemaCompatibility reports an incompatibility. Run that test on every pull request and protect the main branch with the resulting CI check.

Key Takeaways

• Compare every proposed Avro schema with a trusted baseline inside CI.
• Choose backward, forward, or full compatibility from the real producer and consumer deployment order.
• Use Apache Avro's SchemaCompatibility API so local and CI results match.
• Test compatible and deliberately incompatible fixtures to prove the gate itself works.
• Publish precise incompatibility locations so developers can fix failures quickly.
• Keep compatibility checks separate from serialization and broker integration tests.

To validate avro schema compatibility in ci, compare each proposed schema with the last trusted schema and fail the pull request when the selected compatibility policy is violated. This catches contract breaks before a producer publishes data that deployed consumers cannot read.

This tutorial builds that gate with Java 21, Gradle, JUnit 5, and Apache Avro's public compatibility API. It complements the broader Event-Driven API Testing Complete Guide, which explains where schema checks fit beside broker, consumer, retry, and observability tests.

You will create a small repository layout, implement backward and forward checks, prove the checker with breaking fixtures, and run it in GitHub Actions. The same test design works in GitLab CI, Jenkins, or another runner because the decision stays in ordinary Gradle tests.

What You Will Build

By the end, you will have:

• An executable Avro compatibility test using SchemaCompatibility.
• A released baseline schema and a proposed current schema.
• Positive and negative tests for backward, forward, and full compatibility.
• Failure output that identifies the incompatible schema location.
• A GitHub Actions job that blocks incompatible pull requests.

The example models an OrderCreated event. Version 1 is the deployed contract. Version 2 adds an optional field with a default, which is backward compatible. A separate breaking fixture adds a required field without a default so you can verify that the CI gate really fails.

Prerequisites

Use Java 21 and Gradle 8.10 or newer. Apache Avro 1.12.x and JUnit Jupiter 5.11.x are suitable current dependency lines for this example. Check installed versions first:

java -version
gradle --version

Create an empty Git repository and the schema folders:

mkdir avro-compatibility-ci && cd avro-compatibility-ci
git init
mkdir -p src/test/java/com/qajobfit/contracts
mkdir -p src/test/resources/avro/baseline
mkdir -p src/main/avro
mkdir -p src/test/resources/avro/fixtures

You need no Kafka broker or Schema Registry for this test. Compatibility is a deterministic comparison between Avro schemas. Add broker-level tests later when you need to prove registration permissions, subject naming, or actual message transport.

Step 1: Choose the Avro Compatibility Policy

Compatibility direction depends on which side deploys first. In Avro terminology, the reader schema reads data written with the writer schema. The ordering of arguments therefore matters.

For a backward check, call checkReaderWriterCompatibility(newSchema, oldSchema). For a forward check, reverse the arguments. Full compatibility means both calls must succeed.

Start with backward compatibility when consumers are upgraded before producers or when new code must replay historical records. Use full compatibility when producers and consumers roll independently and old instances can remain active. Do not select a policy only because it sounds stricter. Document the deployment scenario it protects.

Verification: Write the policy into the repository README or pull request template in one sentence, such as: OrderCreated requires full compatibility because producer and consumer versions overlap during rolling deployments. Your team should be able to identify the reader and writer for both directions.

Step 2: Configure Gradle and Apache Avro

Create settings.gradle.kts:

rootProject.name = "avro-compatibility-ci"

Create build.gradle.kts:

plugins {
    java
}

repositories {
    mavenCentral()
}

dependencies {
    testImplementation("org.apache.avro:avro:1.12.0")
    testImplementation(platform("org.junit:junit-bom:5.11.4"))
    testImplementation("org.junit.jupiter:junit-jupiter")
}

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }
}

tasks.test {
    useJUnitPlatform()
    testLogging {
        events("passed", "skipped", "failed")
        showStandardStreams = true
        exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
    }
}

Generate and commit the wrapper so CI does not depend on a globally installed Gradle version:

gradle wrapper --gradle-version 8.10.2
./gradlew test

The Avro compiler plugin is unnecessary here because the test parses JSON schemas directly. Generated Java classes can still exist in a production project, but schema compatibility should not depend on code generation succeeding. Keeping the comparison close to raw .avsc files produces clearer contract failures.

Verification: The command ends with BUILD SUCCESSFUL, and Gradle creates build/reports/tests/test/index.html. At this point there are no tests, but dependency resolution and the Java toolchain are working.

Step 3: Add Baseline and Proposed Avro Schemas

Create src/test/resources/avro/baseline/order-created.avsc as the released version:

{
  "type": "record",
  "name": "OrderCreated",
  "namespace": "com.qajobfit.events",
  "fields": [
    { "name": "orderId", "type": "string" },
    { "name": "customerId", "type": "string" },
    { "name": "totalCents", "type": "long" }
  ]
}

Create src/main/avro/order-created.avsc as the proposal:

{
  "type": "record",
  "name": "OrderCreated",
  "namespace": "com.qajobfit.events",
  "fields": [
    { "name": "orderId", "type": "string" },
    { "name": "customerId", "type": "string" },
    { "name": "totalCents", "type": "long" },
    { "name": "couponCode", "type": ["null", "string"], "default": null }
  ]
}

The union order and default agree: the default null matches the first union branch. A new reader can read old data because it supplies null when couponCode is absent. An old reader also ignores a field it does not know, so this particular change passes both directions.

Treat the baseline as immutable evidence of the deployed contract. Update it only after the new schema is released, preferably in a separate release automation step. If a pull request changes both baseline and proposal freely, an author can accidentally compare a breaking schema with itself and bypass the gate.

Verification: Parse both files with Avro's command-line tooling or simply proceed to the parser test in the next step. Confirm that filenames match exactly and that the record name plus namespace remain com.qajobfit.events.OrderCreated.

Step 4: Implement the Apache Avro Compatibility Checker

Create src/test/java/com/qajobfit/contracts/AvroCompatibility.java:

package com.qajobfit.contracts;

import java.io.IOException;
import java.nio.file.Path;
import org.apache.avro.Schema;
import org.apache.avro.SchemaCompatibility;
import org.apache.avro.SchemaCompatibility.SchemaPairCompatibility;

final class AvroCompatibility {
    private AvroCompatibility() {}

    static Schema read(Path path) throws IOException {
        return new Schema.Parser().parse(path.toFile());
    }

    static SchemaPairCompatibility backward(Schema oldSchema, Schema newSchema) {
        return SchemaCompatibility.checkReaderWriterCompatibility(
                newSchema, oldSchema);
    }

    static SchemaPairCompatibility forward(Schema oldSchema, Schema newSchema) {
        return SchemaCompatibility.checkReaderWriterCompatibility(
                oldSchema, newSchema);
    }

    static String describe(SchemaPairCompatibility result) {
        if (result.getType() == SchemaCompatibility.SchemaCompatibilityType.COMPATIBLE) {
            return "compatible";
        }
        return result.getResult().getIncompatibilities().stream()
                .map(item -> item.getType() + " at " + item.getLocation()
                        + ": " + item.getMessage())
                .reduce((left, right) -> left + System.lineSeparator() + right)
                .orElse(result.getDescription());
    }
}

This wrapper gives argument names that expose intent. It also turns Avro's structured incompatibilities into CI-friendly text. Avoid a helper named only compatible(a, b), because future maintainers can easily reverse reader and writer without noticing.

Schema.Parser validates syntax and name references. SchemaCompatibility then applies Avro reader and writer resolution rules, including defaults, unions, primitive promotion, records, enums, and fixed types. This is more reliable than comparing schema JSON or maintaining a custom list of allowed edits.

Verification: Run ./gradlew test. Compilation should succeed. If getIncompatibilities() or another method is missing, inspect the resolved Avro dependency with ./gradlew dependencyInsight --dependency avro --configuration testRuntimeClasspath; an older transitive version may have won resolution.

Step 5: Validate Avro Schema Compatibility in CI Tests

Create src/test/java/com/qajobfit/contracts/OrderCreatedCompatibilityTest.java:

package com.qajobfit.contracts;

import static org.apache.avro.SchemaCompatibility.SchemaCompatibilityType.COMPATIBLE;
import static org.junit.jupiter.api.Assertions.assertEquals;

import java.nio.file.Path;
import org.apache.avro.Schema;
import org.apache.avro.SchemaCompatibility.SchemaPairCompatibility;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;

class OrderCreatedCompatibilityTest {
    private static Schema oldSchema;
    private static Schema newSchema;

    @BeforeAll
    static void loadSchemas() throws Exception {
        oldSchema = AvroCompatibility.read(Path.of(
                "src/test/resources/avro/baseline/order-created.avsc"));
        newSchema = AvroCompatibility.read(Path.of(
                "src/main/avro/order-created.avsc"));
    }

    @Test
    void proposedSchemaIsBackwardCompatible() {
        SchemaPairCompatibility result =
                AvroCompatibility.backward(oldSchema, newSchema);
        assertEquals(COMPATIBLE, result.getType(),
                () -> AvroCompatibility.describe(result));
    }

    @Test
    void proposedSchemaIsForwardCompatible() {
        SchemaPairCompatibility result =
                AvroCompatibility.forward(oldSchema, newSchema);
        assertEquals(COMPATIBLE, result.getType(),
                () -> AvroCompatibility.describe(result));
    }
}

Run the focused test:

./gradlew test --tests '*OrderCreatedCompatibilityTest'

Both methods are necessary for a full policy. If your policy is backward only, retain only the backward test and name the policy explicitly. A clear test name makes a branch protection failure understandable without opening the source.

This check is fast and does not depend on network access after dependencies are cached. It belongs early in the pull request pipeline. Keep a later integration test for registry configuration because a local compatibility pass cannot prove that a registry subject uses the intended compatibility level.

Verification: Gradle reports two passed tests. Now temporarily change couponCode to a plain string without a default. The backward test must fail with an incompatibility such as a missing default value. Restore the compatible schema before continuing.

Step 6: Prove Avro Breaking Change Detection With Fixtures

A green test is trustworthy only when you have demonstrated that it rejects a real break. Create src/test/resources/avro/fixtures/order-created-breaking.avsc:

{
  "type": "record",
  "name": "OrderCreated",
  "namespace": "com.qajobfit.events",
  "fields": [
    { "name": "orderId", "type": "string" },
    { "name": "customerId", "type": "string" },
    { "name": "totalCents", "type": "long" },
    { "name": "salesChannel", "type": "string" }
  ]
}

Add this test method to OrderCreatedCompatibilityTest and import assertNotEquals:

@Test
void detectsRequiredFieldWithoutDefault() throws Exception {
    Schema breakingSchema = AvroCompatibility.read(Path.of(
            "src/test/resources/avro/fixtures/order-created-breaking.avsc"));

    SchemaPairCompatibility result =
            AvroCompatibility.backward(oldSchema, breakingSchema);

    assertNotEquals(COMPATIBLE, result.getType(),
            "The test fixture must remain incompatible");
}

This negative control prevents subtle checker regressions. For example, it catches reversed arguments when the chosen fixture breaks only one direction. Build a compact fixture matrix for changes your team commonly makes: remove an enum symbol, change a field type incompatibly, rename a field without an alias, or change a fixed size.

Do not assert the complete error message because wording can change between library releases. Assert the compatibility type, then print detailed diagnostics only for the policy tests that protect production schemas.

Verification: Run ./gradlew test. All three tests pass. Then add a default to salesChannel; the negative test should fail because the supposedly breaking fixture became compatible. Undo that edit and commit the fixture as a permanent test of the gate.

Step 7: Add the Avro Schema Evolution CI Pipeline

Create .github/workflows/avro-compatibility.yml:

name: Avro compatibility

on:
  pull_request:
    paths:
      - 'src/main/avro/**'
      - 'src/test/resources/avro/**'
      - 'src/test/java/com/qajobfit/contracts/**'
      - 'build.gradle.kts'
      - 'gradle/**'
      - 'gradlew'
      - '.github/workflows/avro-compatibility.yml'
  workflow_dispatch:

permissions:
  contents: read

jobs:
  compatibility:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: '21'
          cache: gradle
      - name: Validate Gradle wrapper
        uses: gradle/actions/wrapper-validation@v6
      - name: Run Avro compatibility tests
        run: ./gradlew test --tests '*CompatibilityTest' --no-daemon
      - name: Upload test report on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: avro-compatibility-test-report
          path: build/reports/tests/test
          if-no-files-found: ignore

Commit the Gradle wrapper, including gradle-wrapper.jar. Configure branch protection so the compatibility job is required. The path filter saves work on unrelated pull requests, while workflow_dispatch lets maintainers rerun the gate after dependency or runner changes.

If baseline schemas come from an artifact repository or registry, retrieve a pinned released version. Never compare against an unpinned latest value because the same commit could pass today and fail tomorrow. Also avoid placing registry credentials in pull request jobs from forks. A committed baseline is simple, reviewable, and safe for many teams.

Verification: Open a test pull request with the compatible schema and confirm the required check passes. In a second commit, remove the couponCode default or point the policy test at the breaking fixture. Confirm the job fails and the uploaded HTML report contains the incompatibility. Revert the deliberate break.

Step 8: Scale the Gate Across Multiple Schemas

One test class per event is explicit, but a parameterized test reduces repetition when a repository contains many schemas. Add a manifest that maps baseline and current files, then iterate over it. Keep event identity visible in the test display name.

A simple JUnit pattern looks like this:

@ParameterizedTest(name = "{0} remains backward compatible")
@CsvSource({
    "order-created, src/test/resources/avro/baseline/order-created.avsc, src/main/avro/order-created.avsc",
    "order-cancelled, src/test/resources/avro/baseline/order-cancelled.avsc, src/main/avro/order-cancelled.avsc"
})
void schemasRemainBackwardCompatible(
        String event, String baselinePath, String currentPath) throws Exception {
    Schema oldSchema = AvroCompatibility.read(Path.of(baselinePath));
    Schema newSchema = AvroCompatibility.read(Path.of(currentPath));
    SchemaPairCompatibility result =
            AvroCompatibility.backward(oldSchema, newSchema);
    assertEquals(COMPATIBLE, result.getType(),
            () -> event + ": " + AvroCompatibility.describe(result));
}

Import org.junit.jupiter.params.ParameterizedTest and org.junit.jupiter.params.provider.CsvSource. The junit-jupiter aggregate dependency already includes parameter support. Prefer an explicit manifest over pairing files only by directory scan, because it makes renamed or deleted schemas visible in review.

For transitive compatibility, compare the proposal against every retained released version, not only the immediately previous one. This matters when consumers skip releases or replay old data. Keep a versioned baseline directory such as baseline/order-created/v1.avsc and v2.avsc, then feed every supported writer schema into the parameterized test.

Verification: Add the second schema files before enabling the second CSV row. Run ./gradlew test --tests '*CompatibilityTest' and confirm each event appears separately in the HTML report. Deliberately break one event and verify that the failure names that event rather than only the generic parameterized method.

Troubleshooting

Problem: A new nullable field still fails -> Put null first in the union when the default is null, and include "default": null. In Avro, a field default must conform to its schema rules and is used by readers when writer data lacks that field.

Problem: The check passes in the wrong direction -> Label variables oldSchema and newSchema, then remember that the API accepts reader first and writer second. Backward means new reader with old writer. Add a one-direction negative fixture to expose argument reversal.

Problem: Local tests pass but Schema Registry rejects the schema -> Check the registry subject, compatibility mode, and whether it enforces transitive compatibility. Local pairwise tests cannot see server configuration or older registered versions that were not included as baselines.

Problem: Named types cannot be resolved -> Parse related schemas with a shared Schema.Parser or parse a protocol that contains the named definitions. Ensure namespaces and aliases are correct. Independently parsing a record that references an external named type can fail before compatibility is evaluated.

Problem: CI cannot execute gradlew -> Commit the wrapper files and run chmod +x gradlew, then commit the executable bit. Keep wrapper validation enabled to detect unexpected wrapper changes.

Problem: A field rename breaks despite identical types -> Avro resolves record fields by name. Add the old name as an alias on the new reader field if the evolution and deployment policy support it, then test both directions. Do not assume matching position makes a rename safe.

Best Practices

• Keep released baselines protected and review their updates separately from proposals.
• Test the compatibility direction your deployment process actually needs.
• Add negative fixtures so the checker cannot become a permanently green decoration.
• Use exact released versions for transitive checks and retain them for the consumer support window.
• Print incompatibility type, location, and message in CI failures.
• Run serialization round trips in addition to structural compatibility checks.
• Test registry integration separately when subjects, authorization, and server policy matter.
• Require the CI job in branch protection instead of relying on developers to read optional checks.

A compatibility API answers whether Avro reader and writer resolution can succeed. It does not prove business semantics. Changing totalCents to contain dollars while retaining type long is structurally compatible and behaviorally disastrous. Pair schema checks with domain assertions, consumer contract tests, and monitoring.

Where To Go Next

Place this gate in a layered event-testing strategy. Review the complete event-driven API testing guide to map schema, producer, broker, consumer, retry, and observability coverage.

Next, follow the Kafka consumer contract testing tutorial to prove that consumer code handles real event fixtures, not only structurally compatible schemas. Then test operational failure behavior with the dead-letter queue retry testing guide. If your system uses AMQP, the RabbitMQ event mocking tutorial shows how to isolate integrations while keeping event payloads realistic.

Use compatibility tests as the earliest gate, serialization tests as the next layer, and broker-backed tests for wiring and infrastructure behavior. This progression keeps pull request feedback fast without mistaking a schema comparison for complete end-to-end assurance.

Interview Questions and Answers

Q: What does backward compatibility mean for an Avro schema?

A new reader schema is backward compatible when it can read data written with the old writer schema. A common safe change is adding a field with a valid default because the new reader can supply that value for historical records.

Q: Why is the argument order important in SchemaCompatibility?

The API accepts reader schema first and writer schema second. Reversing them changes a backward check into a forward check, so descriptive wrapper methods and directional fixtures prevent silent policy mistakes.

Q: Is adding an optional field always compatible?

No. The reader needs a valid default when old records lack the field, and union defaults must conform to Avro rules. Compatibility also depends on direction and the surrounding named types.

Q: What is transitive compatibility?

Transitive compatibility compares a proposed schema against all relevant previous versions, not only the latest one. It protects consumers that skip releases and workloads that replay older retained data.

Q: Why test compatibility locally if Schema Registry checks it?

A local test gives fast, deterministic pull request feedback without credentials or network dependencies. Registry integration tests remain valuable for subject naming, authentication, configured policy, and the actual registered history.

Q: Can schema compatibility guarantee consumer behavior?

No. It validates Avro resolution rules, not business meaning or application logic. Use serialization, consumer contract, and domain behavior tests as additional layers.

Q: How would you make the CI gate trustworthy?

Protect baseline schemas, use the official Avro API, add compatible and incompatible fixtures, print actionable diagnostics, and require the job in branch protection. For long-lived events, compare against every supported released version.

Common Mistakes

The most dangerous mistake is updating the baseline and proposal together until the test passes. That removes the historical contract you intended to protect. Another common error is checking only backward compatibility while claiming full compatibility. Make each direction a separately named test.

Avoid raw JSON diffs as a compatibility decision. Formatting, documentation, and field order changes can create noisy diffs, while subtle resolution breaks need Avro-aware analysis. Also avoid treating a successful schema check as proof that messages can be produced, registered, deserialized by application code, and processed correctly.

Finally, do not leave the check optional. A well-written workflow that is not required can still be merged red. Combine a required status check with code ownership for baseline directories and a documented release update process.

Conclusion: Validate Avro Schema Compatibility in CI\n

To validate avro schema compatibility in ci, preserve a trusted released baseline, compare it with the proposal using Apache Avro's reader-writer compatibility API, and fail a required pull request check when the selected policy is violated. Directional tests, negative fixtures, and precise diagnostics turn that basic comparison into a dependable engineering control.

Run the compatibility suite before expensive integration tests, then extend coverage with serialization, consumer contracts, registry integration, and broker behavior. Start by committing the compatible OrderCreated example, prove the breaking fixture is rejected, and make the GitHub Actions job required on your main branch.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Building evals in CI with promptfoo (2026)
• How to Debug a failing test in VS Code in Cypress (2026)
• How to Debug a failing test in VS Code in Playwright (2026)
• How to Debug a failing test in VS Code in Selenium (2026)
• How to Reduce flaky tests in a CI pipeline (2026)
• How to Run tests in headed mode in Cypress (2026)