Resource library

QA How-To

Test Slowly Changing Dimensions With SQL (2026)

Learn how to test slowly changing dimensions with SQL using runnable PostgreSQL checks for history, overlaps, current rows, idempotency, and fact joins.

24 min read | 1,969 words

TL;DR

To test slowly changing dimensions with SQL, validate one current row per business key, legal and continuous effective ranges, ordered versions, changed attributes, replay safety, and point-in-time fact joins. Run both clean fixtures and deliberate corruptions so the test suite proves that it can fail for the right reason.

Key Takeaways

  • Test business keys, version order, effective ranges, and current-row rules as separate invariants.
  • Use half-open intervals so a boundary timestamp belongs to exactly one dimension version.
  • Prove tracked changes create one version while unchanged events create none.
  • Replay the same source event to expose duplicate-history and idempotency defects.
  • Resolve facts by business key and event time, then compare the result with the stored surrogate key.
  • Make validation SQL return record-level evidence before it fails the pipeline.
  • Run negative fixtures inside transactions so every detector is proven without polluting the warehouse.

To test slowly changing dimensions with SQL, treat history as temporal invariants, not rows to count. For SCD Type 2, prove each business key has one current version, ranges do not overlap, tracked changes create one version, replays create none, and facts resolve to the version valid at event time.

This PostgreSQL 18.4 tutorial builds those checks for a customer dimension and order facts. You will run positive and negative tests, then package them into a fail-fast script. Use the SQL ETL validation guide for wider source-to-target coverage.

TL;DR

Risk SQL evidence Passing result
Multiple current versions Group by business key with filtered counts Zero business keys returned
Bad temporal bounds Compare effective_to with effective_from Zero invalid intervals
Gaps or overlaps Use LAG(effective_to) in version order Every prior end equals the next start
False history Compare adjacent tracked attributes No consecutive identical states
Non-idempotent replay Apply one event twice and compare counts Second application adds zero rows
Wrong fact surrogate Resolve by business key and event timestamp Stored and expected keys match

The lab uses half-open intervals: effective_from <= event_time and event_time < effective_to. Null effective_to means current. Translate every predicate if your warehouse uses inclusive ends or a sentinel date.

What You Will Build

You will create a disposable SCD Type 2 lab that contains:

  • A dim_customer history table with deterministic business keys, surrogate keys, tracked attributes, version numbers, source event IDs, and effective timestamps.
  • A fact_order table whose stored customer surrogate keys can be checked against point-in-time history.
  • Focused SQL checks for current-row cardinality, flag consistency, range validity, continuity, version order, and redundant versions.
  • Transactional negative tests that inject one defect, prove the detector reports it, and roll back the mutation.
  • A single scd_tests.sql runner that prints useful evidence and exits nonzero when any invariant fails.

These checks return exception rows instead of bare booleans. The failing key, range, and surrogate value help distinguish a loader defect from a fixture or policy mistake without rerunning an unbounded warehouse query.

This lab concentrates on Type 2. Know the expected change strategy before defining an oracle:

SCD strategy Change behavior Primary test oracle
Type 0 Original value never changes First accepted value remains unchanged
Type 1 Row is overwritten Current value matches the latest accepted source state
Type 2 New version preserves history Ranges, versions, current flag, and temporal joins are correct
Type 3 Selected prior value is retained in columns Current and previous columns shift according to the mapping

Do not apply Type 2 expectations to every attribute. A segment may keep history while a corrected spelling is overwritten. List the tracked columns from the source-to-target mapping first.

Prerequisites

Use Docker Engine 27.5.1 or newer and official image postgres:18.4-alpine3.23, which includes server and psql 18.4. You need a POSIX-compatible shell and three empty files: schema.sql, apply_change.sql, and scd_tests.sql.

Confirm Docker is available:

docker version --format '{{.Server.Version}}'

Expected output is 27.5.1 or newer. The SQL needs PostgreSQL 18 and no extensions.

Keep this lab away from production credentials. For reusable environment patterns, see SQL test data setup and teardown.

Step 1: Set Up a Lab to Test Slowly Changing Dimensions With SQL

Start PostgreSQL on host port 54329. The complete image tag keeps CI runs reproducible.

docker run --name scd-postgres-18 \
  -e POSTGRES_PASSWORD=qa_local_only \
  -e POSTGRES_DB=warehouse \
  -p 54329:5432 \
  -d postgres:18.4-alpine3.23

Poll database readiness instead of sleeping for a fixed duration.

until docker exec scd-postgres-18 pg_isready -U postgres -d warehouse; do
  sleep 1
done

The last line should contain accepting connections. Verify the server build:

docker exec scd-postgres-18 \
  psql -U postgres -d warehouse -Atc 'SHOW server_version;'

Expected output starts with 18.4. A refusal means initialization is incomplete or the container stopped.

Step 2: Create a Valid SCD Type 2 Fixture

Save this as schema.sql. customer_id is the business key; customer_sk is the surrogate key. segment and city are tracked.

\set ON_ERROR_STOP on

DROP TABLE IF EXISTS fact_order;
DROP TABLE IF EXISTS dim_customer;

CREATE TABLE dim_customer (
    customer_sk bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    customer_id text NOT NULL,
    full_name text NOT NULL,
    segment text NOT NULL,
    city text NOT NULL,
    effective_from timestamptz NOT NULL,
    effective_to timestamptz,
    is_current boolean NOT NULL,
    version_no integer NOT NULL CHECK (version_no > 0),
    source_event_id bigint NOT NULL UNIQUE
);

CREATE TABLE fact_order (
    order_id bigint PRIMARY KEY,
    customer_id text NOT NULL,
    ordered_at timestamptz NOT NULL,
    customer_sk bigint NOT NULL REFERENCES dim_customer(customer_sk),
    amount numeric(12, 2) NOT NULL
);

INSERT INTO dim_customer (
    customer_sk, customer_id, full_name, segment, city,
    effective_from, effective_to, is_current, version_no, source_event_id
) OVERRIDING SYSTEM VALUE VALUES
    (1001, 'CUST-100', 'Asha Rao', 'Free', 'Pune',
     '2026-01-01T00:00:00Z', '2026-03-15T10:00:00Z', false, 1, 9001),
    (1002, 'CUST-100', 'Asha Rao', 'Pro', 'Mumbai',
     '2026-03-15T10:00:00Z', NULL, true, 2, 9002),
    (2001, 'CUST-200', 'Luis Park', 'Business', 'Delhi',
     '2026-02-01T00:00:00Z', NULL, true, 1, 9003),
    (3001, 'CUST-300', 'Mina Shah', 'Free', 'Chennai',
     '2026-01-10T00:00:00Z', '2026-04-01T00:00:00Z', false, 1, 9004),
    (3002, 'CUST-300', 'Mina Shah', 'Pro', 'Chennai',
     '2026-04-01T00:00:00Z', '2026-06-01T00:00:00Z', false, 2, 9005),
    (3003, 'CUST-300', 'Mina Shah', 'Business', 'Bengaluru',
     '2026-06-01T00:00:00Z', NULL, true, 3, 9006);

ALTER TABLE dim_customer ALTER COLUMN customer_sk RESTART WITH 4000;

INSERT INTO fact_order VALUES
    (5001, 'CUST-100', '2026-02-10T09:00:00Z', 1001, 49.00),
    (5002, 'CUST-100', '2026-03-15T10:00:00Z', 1002, 99.00),
    (5003, 'CUST-300', '2026-05-20T12:00:00Z', 3002, 125.00),
    (5004, 'CUST-300', '2026-06-10T12:00:00Z', 3003, 175.00);

Load the fixture:

docker exec -i scd-postgres-18 \
  psql -U postgres -d warehouse < schema.sql

Verify explicit totals instead of trusting only insert messages:

docker exec scd-postgres-18 psql -U postgres -d warehouse -c \
  "SELECT COUNT(*) AS versions, COUNT(*) FILTER (WHERE is_current) AS current_versions FROM dim_customer;"

Expect six versions and three current versions. Totals validate the fixture, not per-customer history.

Step 3: Validate Current Rows and Flag Consistency

Normally, each active business key has exactly one current row. A filtered aggregate catches zero and multiple current versions.

SELECT
    customer_id,
    COUNT(*) FILTER (WHERE is_current) AS current_count
FROM dim_customer
GROUP BY customer_id
HAVING COUNT(*) FILTER (WHERE is_current) <> 1;

Also test agreement between the flag and the open interval:

SELECT customer_sk, customer_id, is_current, effective_to
FROM dim_customer
WHERE (is_current AND effective_to IS NOT NULL)
   OR (NOT is_current AND effective_to IS NULL);

Verify both checks in one command. Each scalar result must be zero:

docker exec scd-postgres-18 psql -U postgres -d warehouse -Atc \
  "SELECT COUNT(*) FROM (SELECT customer_id FROM dim_customer GROUP BY customer_id HAVING COUNT(*) FILTER (WHERE is_current) <> 1) q; SELECT COUNT(*) FROM dim_customer WHERE (is_current AND effective_to IS NOT NULL) OR (NOT is_current AND effective_to IS NULL);"

Both scalar results must be 0. A negative test can insert a second current row inside a transaction, run the grouped check, and roll back. Add preventive indexes only after legacy data passes; database constraint testing explains why.

Step 4: Detect Invalid Ranges, Gaps, Overlaps, and Version Errors

Reject closed intervals whose end is not after their start. Then order versions per key and compare each start with the prior end. Equality is continuous, a later start is a gap, and an earlier start is an overlap.

WITH sequenced AS (
    SELECT
        customer_sk,
        customer_id,
        version_no,
        effective_from,
        effective_to,
        LAG(effective_to) OVER (
            PARTITION BY customer_id
            ORDER BY effective_from, customer_sk
        ) AS previous_to,
        ROW_NUMBER() OVER (
            PARTITION BY customer_id
            ORDER BY effective_from, customer_sk
        ) AS expected_version
    FROM dim_customer
)
SELECT
    customer_id, customer_sk, version_no, expected_version,
    previous_to, effective_from,
    CASE
        WHEN effective_to IS NOT NULL AND effective_to <= effective_from
            THEN 'INVALID_BOUNDS'
        WHEN expected_version > 1 AND previous_to < effective_from
            THEN 'GAP'
        WHEN expected_version > 1 AND previous_to > effective_from
            THEN 'OVERLAP'
        WHEN version_no <> expected_version
            THEN 'VERSION_SEQUENCE'
    END AS violation
FROM sequenced
WHERE (effective_to IS NOT NULL AND effective_to <= effective_from)
   OR (expected_version > 1 AND previous_to IS DISTINCT FROM effective_from)
   OR version_no <> expected_version;

customer_sk breaks ties deterministically. Learn more from the SQL window functions tutorial.

Verify the clean fixture:

docker exec scd-postgres-18 psql -U postgres -d warehouse -Atc \
  "WITH s AS (SELECT customer_sk, customer_id, version_no, effective_from, effective_to, LAG(effective_to) OVER (PARTITION BY customer_id ORDER BY effective_from, customer_sk) previous_to, ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY effective_from, customer_sk) expected_version FROM dim_customer) SELECT COUNT(*) FROM s WHERE (effective_to IS NOT NULL AND effective_to <= effective_from) OR (expected_version > 1 AND previous_to IS DISTINCT FROM effective_from) OR version_no <> expected_version;"

Expected output is 0. Now run a controlled negative test:

BEGIN;
UPDATE dim_customer
SET effective_from = '2026-03-10T10:00:00Z'
WHERE customer_sk = 1002;

WITH s AS (
    SELECT customer_sk, customer_id, effective_from,
           LAG(effective_to) OVER (
               PARTITION BY customer_id ORDER BY effective_from, customer_sk
           ) AS previous_to
    FROM dim_customer
)
SELECT * FROM s
WHERE previous_to > effective_from;
ROLLBACK;

Expect CUST-100, which proves the detector sees the overlap. Roll back and confirm the clean count returns to zero. If gaps are permitted for returning customers, encode that policy explicitly.

Step 5: Test Tracked Changes, Unchanged Inputs, and Replays

Structural checks do not prove loader behavior. This forward-only function closes the current row and inserts a version only when tracked values differ. A consumed event ID returns zero.

CREATE OR REPLACE FUNCTION apply_customer_change(
    p_event_id bigint,
    p_customer_id text,
    p_full_name text,
    p_segment text,
    p_city text,
    p_changed_at timestamptz
) RETURNS integer
LANGUAGE plpgsql
AS $
DECLARE
    v_next_version integer;
    v_closed integer;
BEGIN
    IF EXISTS (
        SELECT 1 FROM dim_customer WHERE source_event_id = p_event_id
    ) THEN
        RETURN 0;
    END IF;

    SELECT COALESCE(MAX(version_no), 0) + 1
    INTO v_next_version
    FROM dim_customer
    WHERE customer_id = p_customer_id;

    UPDATE dim_customer
    SET effective_to = p_changed_at, is_current = false
    WHERE customer_id = p_customer_id
      AND is_current
      AND effective_from < p_changed_at
      AND ROW(full_name, segment, city)
          IS DISTINCT FROM ROW(p_full_name, p_segment, p_city);

    GET DIAGNOSTICS v_closed = ROW_COUNT;

    IF v_closed = 1 THEN
        INSERT INTO dim_customer (
            customer_id, full_name, segment, city, effective_from,
            effective_to, is_current, version_no, source_event_id
        ) VALUES (
            p_customer_id, p_full_name, p_segment, p_city, p_changed_at,
            NULL, true, v_next_version, p_event_id
        );
    END IF;

    RETURN v_closed;
END;
$;

Save the block as apply_change.sql and pipe it to psql. Production loaders also need concurrency, initial-insert, late-event, and rejection policies.

Exercise three behaviors in one rollback-safe scenario:

BEGIN;

SELECT apply_customer_change(
    9100, 'CUST-200', 'Luis Park', 'Enterprise', 'Delhi',
    '2026-07-01T00:00:00Z'
) AS first_tracked_change;

SELECT apply_customer_change(
    9100, 'CUST-200', 'Luis Park', 'Enterprise', 'Delhi',
    '2026-07-01T00:00:00Z'
) AS replayed_event;

SELECT apply_customer_change(
    9101, 'CUST-200', 'Luis Park', 'Enterprise', 'Delhi',
    '2026-07-02T00:00:00Z'
) AS unchanged_event;

SELECT
    COUNT(*) AS total_versions,
    COUNT(*) FILTER (WHERE is_current) AS current_versions,
    MAX(version_no) AS latest_version
FROM dim_customer
WHERE customer_id = 'CUST-200';

ROLLBACK;

Expect returns of 1, 0, and 0, followed by two versions, one current row, and version 2. This proves change creation, replay safety, and unchanged-event suppression before rollback.

Use LAG to find adjacent versions whose tracked tuples are identical:

WITH states AS (
    SELECT
        customer_sk, customer_id, full_name, segment, city,
        LAG(full_name) OVER w AS previous_name,
        LAG(segment) OVER w AS previous_segment,
        LAG(city) OVER w AS previous_city,
        ROW_NUMBER() OVER w AS position
    FROM dim_customer
    WINDOW w AS (
        PARTITION BY customer_id ORDER BY effective_from, customer_sk
    )
)
SELECT customer_sk, customer_id
FROM states
WHERE position > 1
  AND ROW(full_name, segment, city)
      IS NOT DISTINCT FROM ROW(previous_name, previous_segment, previous_city);

The clean fixture returns nothing. IS DISTINCT FROM gives deliberate null semantics.

Step 6: Verify Point-in-Time Fact Resolution and Boundaries

Recompute each fact's surrogate key from its business key and event time, then compare it with the stored key.

SELECT
    f.order_id,
    f.customer_id,
    f.ordered_at,
    f.customer_sk AS stored_sk,
    expected.customer_sk AS expected_sk
FROM fact_order AS f
LEFT JOIN LATERAL (
    SELECT d.customer_sk
    FROM dim_customer AS d
    WHERE d.customer_id = f.customer_id
      AND d.effective_from <= f.ordered_at
      AND (d.effective_to IS NULL OR f.ordered_at < d.effective_to)
    ORDER BY d.effective_from DESC, d.customer_sk DESC
) AS expected ON true
WHERE f.customer_sk IS DISTINCT FROM expected.customer_sk;

The left lateral join keeps facts with no valid version visible. Run overlap checks separately because ordering could otherwise select one of several matches.

Order 5002 is exactly on a boundary and must resolve to version 1002. Verify all four facts:

docker exec scd-postgres-18 psql -U postgres -d warehouse -Atc \
  "SELECT COUNT(*) FROM fact_order f LEFT JOIN LATERAL (SELECT d.customer_sk FROM dim_customer d WHERE d.customer_id = f.customer_id AND d.effective_from <= f.ordered_at AND (d.effective_to IS NULL OR f.ordered_at < d.effective_to) ORDER BY d.effective_from DESC, d.customer_sk DESC) expected ON true WHERE f.customer_sk IS DISTINCT FROM expected.customer_sk;"

Expect 0. As a negative test, change order 5002 to key 1001 inside a transaction. The query must show stored 1001, expected 1002; then roll back. See validating data integrity with SQL for more referential checks.

Step 7: Automate How You Test Slowly Changing Dimensions With SQL

Save this standardized failure collector as scd_tests.sql. Every finding includes a rule, business key, and diagnostic payload.

\set ON_ERROR_STOP on
BEGIN;

CREATE TEMP TABLE scd_test_failures (
    rule_id text NOT NULL,
    business_key text NOT NULL,
    details text NOT NULL
) ON COMMIT DROP;

INSERT INTO scd_test_failures
SELECT 'CURRENT_COUNT', customer_id,
       format('current_count=%s', COUNT(*) FILTER (WHERE is_current))
FROM dim_customer
GROUP BY customer_id
HAVING COUNT(*) FILTER (WHERE is_current) <> 1;

INSERT INTO scd_test_failures
SELECT 'FLAG_END_AGREEMENT', customer_id,
       format('sk=%s current=%s effective_to=%s',
              customer_sk, is_current, COALESCE(effective_to::text, 'NULL'))
FROM dim_customer
WHERE (is_current AND effective_to IS NOT NULL)
   OR (NOT is_current AND effective_to IS NULL);

WITH sequenced AS (
    SELECT d.*,
           LAG(effective_to) OVER w AS previous_to,
           ROW_NUMBER() OVER w AS expected_version
    FROM dim_customer AS d
    WINDOW w AS (
        PARTITION BY customer_id ORDER BY effective_from, customer_sk
    )
)
INSERT INTO scd_test_failures
SELECT 'TEMPORAL_SEQUENCE', customer_id,
       format('sk=%s version=%s expected=%s previous_to=%s from=%s',
              customer_sk, version_no, expected_version,
              COALESCE(previous_to::text, 'NULL'), effective_from)
FROM sequenced
WHERE (effective_to IS NOT NULL AND effective_to <= effective_from)
   OR (expected_version > 1 AND previous_to IS DISTINCT FROM effective_from)
   OR version_no <> expected_version;

WITH states AS (
    SELECT d.*,
           LAG(full_name) OVER w AS previous_name,
           LAG(segment) OVER w AS previous_segment,
           LAG(city) OVER w AS previous_city,
           ROW_NUMBER() OVER w AS position
    FROM dim_customer AS d
    WINDOW w AS (
        PARTITION BY customer_id ORDER BY effective_from, customer_sk
    )
)
INSERT INTO scd_test_failures
SELECT 'REDUNDANT_VERSION', customer_id,
       format('sk=%s repeats the prior tracked state', customer_sk)
FROM states
WHERE position > 1
  AND ROW(full_name, segment, city)
      IS NOT DISTINCT FROM ROW(previous_name, previous_segment, previous_city);

INSERT INTO scd_test_failures
SELECT 'FACT_POINT_IN_TIME', f.customer_id,
       format('order=%s stored_sk=%s expected_sk=%s',
              f.order_id, f.customer_sk, COALESCE(x.customer_sk::text, 'NULL'))
FROM fact_order AS f
LEFT JOIN LATERAL (
    SELECT d.customer_sk
    FROM dim_customer AS d
    WHERE d.customer_id = f.customer_id
      AND d.effective_from <= f.ordered_at
      AND (d.effective_to IS NULL OR f.ordered_at < d.effective_to)
    ORDER BY d.effective_from DESC, d.customer_sk DESC
) AS x ON true
WHERE f.customer_sk IS DISTINCT FROM x.customer_sk;

TABLE scd_test_failures;
SELECT COUNT(*) AS failure_count FROM scd_test_failures;

DO $
DECLARE v_failures integer;
BEGIN
    SELECT COUNT(*) INTO v_failures FROM scd_test_failures;
    IF v_failures > 0 THEN
        RAISE EXCEPTION 'SCD test suite found % violation(s)', v_failures;
    END IF;
END;
$;

ROLLBACK;

Run it with psql in noninteractive mode:

docker exec -i scd-postgres-18 \
  psql -U postgres -d warehouse < scd_tests.sql

A valid fixture prints failure_count = 0 and exits zero. Any finding is printed before the exception, while ON_ERROR_STOP gives CI a nonzero status.

Prove the complete runner with a negative fixture:

docker exec scd-postgres-18 psql -U postgres -d warehouse -c \
  "UPDATE dim_customer SET version_no = 7 WHERE customer_sk = 3002;"

docker exec -i scd-postgres-18 \
  psql -U postgres -d warehouse < scd_tests.sql

# Restore the deterministic fixture after the expected failure.
docker exec -i scd-postgres-18 \
  psql -U postgres -d warehouse < schema.sql

The middle command must fail with TEMPORAL_SEQUENCE. Inject defects only in isolated data. The ephemeral database tutorial provides stronger automation isolation.

Troubleshooting

Problem: docker: Error response from daemon: Conflict names scd-postgres-18. -> Fix: A container with that name already exists. Inspect it with docker ps -a --filter name=scd-postgres-18. Start the stopped lab with docker start scd-postgres-18, or remove only that known disposable container after confirming it holds no needed work.

Problem: A gap query reports every transition. -> Fix: Check timestamp types, precision, timezone normalization, and interval convention. A loader that subtracts one second from an inclusive end date will not satisfy half-open equality. Rewrite the oracle for the documented model instead of adding a tolerance that could hide a real missing period.

Problem: The current-count query returns zero for deleted customers. -> Fix: Decide how deletion is represented. If deletion closes the final row with no active replacement, scope the exactly-one-current rule to active business keys and add a separate deletion-state assertion. Do not exclude every zero-current key without evidence from the source deletion feed.

Problem: Point-in-time lookup returns more than one dimension row. -> Fix: Run the overlap detector first and preserve all matching surrogate keys. ORDER BY ... LIMIT 1 makes a query deterministic but does not make overlapping history correct. Repair the ranges or document a source precedence rule.

Problem: Replay testing creates a duplicate version. -> Fix: Verify that the loader records an immutable event ID or another stable deduplication key in the same transaction as the dimension change. A hash of mutable payload formatting is weak because harmless serialization differences can defeat it. Test retry after a simulated failure at each transaction boundary.

Problem: The SQL passes locally but times out on the warehouse. -> Fix: Restrict validation to a closed load batch or affected business-key set, then keep a scheduled full-history audit. Index or cluster by business key and effective start where the engine supports it. Review the execution plan before changing predicates, because a faster query that drops unmatched facts is not equivalent.

Interview Questions and Answers

The interview panel below covers temporal invariants, negative fixtures, replay safety, and fact resolution. State the interval convention and business-key grain before presenting SQL.

Common Mistakes

  • Checking only the latest load batch, which misses broken historical ranges created by earlier runs.
  • Comparing current-row totals across the table instead of grouping by the complete business key.
  • Using BETWEEN for both range boundaries, which makes the shared transition instant match two versions.
  • Treating a null end timestamp and a distant sentinel date as interchangeable without changing predicates.
  • Validating surrogate-key uniqueness while ignoring duplicate business-key versions.
  • Copying the loader's change-detection expression into the oracle, allowing the same defect to exist in both.
  • Testing one successful update but never testing unchanged input, duplicate delivery, out-of-order events, or late facts.
  • Joining facts to the current dimension row instead of the row effective at the fact's event time.
  • Failing on a count without printing customer IDs, surrogate keys, timestamps, and rule names.

Separate universal rules from model policies. Positive duration is usually universal; continuity and current-row expectations can vary for deletions or multi-source dimensions. Use the data migration testing guide for backfill risks.

Where To Go Next

Move passing queries to a read-only role and run them against an immutable batch, clone, or snapshot. Retain exception rows, trend rule IDs, and schedule a full audit outside incremental windows.

Continue with these verified QAJobFit resources:

Delete the disposable lab with docker rm -f scd-postgres-18 only after confirming it contains nothing to preserve.

Conclusion

To test slowly changing dimensions with SQL well, combine temporal invariants with transition scenarios. Structural queries expose corrupt history; change, replay, and point-in-time tests prove behavior.

Run the clean fixture, inject one defect, and require actionable nonzero failure. Then adapt tracked columns, deletion, interval, late-arrival, and batch policies to your warehouse.

Interview Questions and Answers

Which invariants would you test first in an SCD Type 2 dimension?

I would start with the declared business-key grain, exactly one current version for each active key, agreement between current flags and open ends, positive interval duration, and nonoverlapping history. I would then check consecutive version numbering and confirm adjacent rows differ in at least one tracked attribute. Each query should return offending keys rather than only a total.

Why are half-open time intervals useful for slowly changing dimensions?

A half-open interval includes `effective_from` and excludes `effective_to`. When one version ends at the same instant the next begins, an event at that instant matches only the newer version. This avoids the double match produced by inclusive predicates on both ends.

How would you test that an unchanged source event does not create history?

I would capture the row count, current surrogate key, version number, and effective start for one business key. After processing an event whose tracked attributes are identical, all four observations must remain unchanged. I would still verify any separately specified audit behavior instead of assuming the event disappears completely.

How do late-arriving facts differ from late-arriving dimension changes?

A late fact usually needs a point-in-time lookup against history that already exists. A late dimension change may require splitting a closed interval, shifting later boundaries, and deciding whether version numbers are stable or recomputed. I test those policies separately because their update surfaces and recovery risks are different.

How would you diagnose two current rows for one business key?

I would retrieve both surrogate keys, their source event IDs, load batches, effective starts, and transaction timestamps. Then I would check concurrent processing, retry deduplication, and whether the close-and-insert operation was atomic. The repair should follow an owned precedence rule, not whichever row happens to have the larger surrogate key.

Why is joining a fact to the current dimension row incorrect?

The current row describes the entity now, while the fact records a business event at an earlier time. Historical reporting needs the version effective at the fact timestamp so measures retain their original dimensional context. I compare the stored surrogate key with an independently resolved temporal key and include exact-boundary cases.

How would you scale SCD validation on a very large warehouse?

I would run focused checks for business keys touched by each closed batch and preserve rule-level exceptions. A scheduled full-history audit would cover defects outside incremental windows, with partition pruning, clustering, or indexes aligned to business key and effective time. I would compare query plans before optimizing to ensure unmatched or overlapping rows remain observable.

Frequently Asked Questions

How do you test an SCD Type 2 table with SQL?

Group by the business key to verify current-row cardinality, inspect effective bounds, use window functions to detect gaps and overlaps, and compare adjacent tracked states. Then exercise the loader with a changed event, unchanged event, replay, boundary timestamp, and late fact so structural checks and behavioral checks cover different risks.

How can SQL detect overlapping SCD date ranges?

Order versions by effective start within each business key and compare the current start with `LAG(effective_to)`. Under half-open intervals, `previous_to > effective_from` indicates an overlap, while equality indicates a clean transition.

Should an SCD Type 2 dimension always have one current row?

Usually each active business key has exactly one current version, but deleted entities may legitimately have none if deletion closes history. Define the deletion contract first, scope the current-row rule to active keys, and test deleted keys with their own assertion.

What is the safest effective-date convention for SCD Type 2?

Half-open intervals are practical because the start is included and the end is excluded. Adjacent versions can share one boundary timestamp without both matching a fact, provided every loader and validation query uses the same predicates.

How do you test SCD Type 2 idempotency?

Apply one source event, record the version count and current state, then replay the identical immutable event ID. The second application must add no row, close no interval, and leave the resolved current surrogate key unchanged.

How do you validate a fact table against SCD history?

Resolve the expected dimension row using the fact's business key and event time, then compare that surrogate key with the one stored on the fact. Use a left lateral join or equivalent so facts with no matching historical version remain visible as failures.

Can database constraints replace SCD validation queries?

Constraints can prevent some invalid states, such as duplicate event IDs or impossible version numbers, but cross-row temporal rules often need exclusion constraints, loader controls, or audit SQL. Keep diagnostic queries because they provide business keys and timestamps needed to repair existing data.

Related Guides