Deequ with Apache Spark
bCloud LLC · Software Development
Certification per Microsoft Marketplace.
Evidence tier Source Confirmed · 9 captures on record
What the publisher says
As described on Microsoft Marketplace.
Deequ with Apache Spark is a data quality framework developed by Amazon that runs on top of Apache Spark. It allows organizations to define, automate, and enforce data quality checks on large datasets, leveraging Spark’s distributed computing for scalable and efficient analysis. Deequ helps ensure the reliability, consistency, and completeness of data in production pipelines.
Features of Deequ with Apache Spark:
Show the rest of the publisher’s description (13 more lines)
- Data quality framework: Enables definition of constraints and metrics such as completeness, uniqueness, and consistency to validate data quality.
- Distributed computation: Utilizes Apache Spark’s distributed processing for scalable verification of large datasets.
- Historical tracking: Supports storing verification results over time to monitor trends in data quality.
- Declarative API: Allows users to define checks and analysis in a concise and readable manner, while integrating seamlessly into Spark pipelines.
- Compatible with Ubuntu 24.04 and Python virtual environments for isolated PyDeequ package management.
Deequ Usage with PySpark:
$ sudo su
$ source /opt/pydeequ-venv/bin/activate
$ /opt/spark/bin/pyspark --packages com.amazon.deequ:deequ:2.0.7-spark-3.5
$Once Spark starts successfully, test:
$ import pydeequ
print(pydeequ.__version__)
Disclaimer: Deequ is an independent open-source project developed by Amazon and is not affiliated with, endorsed by, or sponsored by Apache Spark or any Linux distribution.
Agent build and provenance
See the full provenance
The layer-by-layer build, the evidence behind each claim, the risk basis and the cross-marketplace links are open to any account. Some rows are disclosed, some the source leaves Unknown; a free account shows you which.
Compliance
- FedRAMPConfirmedNot listed90%, registry-checkedNo FedRAMP Marketplace entry matched this vendor's domain, checked 2026-08-27registry recordas observed 2026-08-27
Confirmed means matched to a public authoritative registry. Claimed means the vendor or its listing states it, not yet cross-checked. A framework not shown was not found in any source we hold, which is not evidence against it. Not listed means a scoped registry check found no match for this vendor's domain: a No is a scoped registry check, not a compliance judgment. Confidence bands: 95% domain-verified, 90% registry-checked, 80% self-attested, 70% weak signal. Self-attested items marked “vendor's site” are gathered from the vendor's own website and are not verified by us.
Vendor
External enrichment · as of 2026-08-29
Sources
Publisher resources
1 linkLinked repositories
Unknown means this listing does not publish a repository. It is not a statement that the code is closed, and a linked repository is not a claim that the publisher wrote it: the registry computes that relationship privately and does not publish it.
Evidence risk is the share of the build you cannot see before you deploy, not a security rating. Sign in to see the layer-by-layer basis for this band.