by Ada Sharoni, Ariel Krinitsi, Wojciech Indyk, Irach Ramos Monitoring Data Quality is becoming a frequent requirement in any data pipeline, including Spark based applications. Suppose we want to make sure our data comes in certain shape and size, and be alerted when certain criteria set is not met. Why…