Skip to main content

Clean

The Clean module standardizes raw data at scale so downstream matching compares like with like.

Why Data Cleaning Matters​

Healthcare data arrives in inconsistent formats across sources. That fragments patient records, causes missed matches, skews analytics, and creates compliance risk.

What You Can Do​

Apply Cleaning Rules

Configure rules for whitespace, case, phone formats, and more.

Turn Rules On and Off

Disable a rule for the next run without losing its configuration.

Chain Multiple Operations

Apply several operations in sequence.

Track Execution

Monitor jobs in real time with logs.

Clean Several Sources at Once

Pick multiple source tables and one destination schema. Each table is written there as <table>_cleaned, with its own saved rules applied.

Inspect Rule Results

See the columns each rule touched and how many rows it changed.

Cleaning Operations​

Text Normalization

Trim whitespace, convert case, remove special characters

Phone Standardization

Strip formatting, handle country codes, apply a consistent pattern

Date Standardization

Read many input formats and write one standard format

Name Standardization

Title-case names and collapse stray whitespace

Address Standardization

Title-case addresses and expand street-type and directional abbreviations

Value Replacement

Replace values, fill nulls, and apply custom regular expressions

Key Capabilities​

Rule-Based Configuration​

Define rules visually without code: select columns, choose operations, set parameters.

Scalable Processing​

Jobs run natively on Databricks (Spark), Snowflake (Snowpark), or Microsoft Fabric (OneLake), without moving data.

Rule Conflict Detection​

Conflicting case transformations or duplicate rules on a column are flagged while you configure them, not at run time.

Run History and Recovery​

A live run window shows progress and row counts and keeps past runs. Failed or lost runs are reconciled automatically so the stage never sticks in "running".

Column Profiling​

After cleaning, each column is profiled for nulls, unique values, and data types.

Business Impact​

Higher Match Rates

Standardized data improves matching accuracy.

Reduced Manual Work

Automate repetitive cleanup.

Trusted Analytics

Clean data yields reliable reports.

Who Benefits​

  • Data Stewards: Maintain quality standards
  • Clinical Teams: Access accurate patient information
  • Revenue Cycle: Reduce data-driven claim denials
  • Compliance Officers: Meet data accuracy requirements