Deequ with Apache Spark
bCloud LLC · Software Development
Certification per Microsoft Marketplace.
Evidence tier Source Confirmed · 10 captures on record
What the publisher says
As described on Microsoft Marketplace.
Deequ with Apache Spark is an open-source data quality and validation framework developed by Amazon that runs on top of Apache Spark. It enables organizations to define, measure, and verify data quality constraints on large-scale datasets using Spark's distributed processing capabilities. Deequ helps identify data anomalies, enforce quality rules, and automate validation in data engineering and analytics pipelines.
Features of Deequ with Apache Spark:
Show the rest of the publisher’s description (12 more lines)
- Data quality validation: Defines constraints such as completeness, uniqueness, accuracy, consistency, and validity to ensure reliable datasets.
- Distributed processing: Leverages Apache Spark's distributed computing engine to efficiently analyze large datasets.
- Constraint verification: Automatically verifies data against predefined quality checks and reports violations.
- Metrics repository: Stores computed data quality metrics for historical comparison and trend analysis.
- Seamless Spark integration: Integrates directly with Spark applications using Scala, Java, and PyDeequ for Python-based workflows.
- Scalable architecture: Supports batch processing on local clusters, Hadoop, Kubernetes, and cloud-based Spark deployments.
Deequ Usage with Apache Spark:
$ sudo su
$ cd /opt
$ source /opt/pydeequ-venv/bin/activate
$ /opt/spark/bin/pyspark --packages com.amazon.deequ:deequ:2.0.7-spark-3.5
Disclaimer: Deequ is an independent open-source project developed by Amazon Web Services (AWS). Apache Spark is a separate open-source project of the Apache Software Foundation. Deequ is not affiliated with, endorsed by, or sponsored by the Apache Software Foundation or any Linux distribution.
Agent build and provenance
See the full provenance
The layer-by-layer build, the evidence behind each claim, the risk basis and the cross-marketplace links are open to any account. Some rows are disclosed, some the source leaves Unknown; a free account shows you which.
Compliance
- FedRAMPConfirmedNot listed90%, registry-checkedNo FedRAMP Marketplace entry matched this vendor's domain, checked 2026-08-27registry recordas observed 2026-08-27
Confirmed means matched to a public authoritative registry. Claimed means the vendor or its listing states it, not yet cross-checked. A framework not shown was not found in any source we hold, which is not evidence against it. Not listed means a scoped registry check found no match for this vendor's domain: a No is a scoped registry check, not a compliance judgment. Confidence bands: 95% domain-verified, 90% registry-checked, 80% self-attested, 70% weak signal. Self-attested items marked “vendor's site” are gathered from the vendor's own website and are not verified by us.
Vendor
External enrichment · as of 2026-08-29
Sources
Publisher resources
1 linkLinked repositories
Unknown means this listing does not publish a repository. It is not a statement that the code is closed, and a linked repository is not a claim that the publisher wrote it: the registry computes that relationship privately and does not publish it.
Evidence risk is the share of the build you cannot see before you deploy, not a security rating. Sign in to see the layer-by-layer basis for this band.