Karan RewariRunning (Py)Spark on Airflow Locally & Processing Good/Bad Data separately (Batch-mode).Tech Stack: Docker, Airflow, Spark, S33 min read·May 8, 2021----
Karan RewariValidation Layer: An approach to schema validation for Big DataFor numerous reasons, having a validation layer in a data platform is critical, with the end goal of having control over what and how data…6 min read·May 16, 2020----