The purpose of this script is to provide an interface for data scientists to interact with parameters submitted via an HTML form. By defining a schema at the top of a python script, the named parameters will either be the correct type and validated against regex, or the program will fail at the validation step. The schema can also be returned as JSON so a consumer of an API can understand what is required.
It also provides methods to read and write from mysql or psql, and write results to AWS buckets via Boto 3.
When writing this I was especially enthusiastic about list comprehensions and anonymous functions so this might not be an example of doing things the most “pythonic” way, I am a javascript programmer at heart.
A TODO for this project is to create an endpoint that builds the web form based on the schema of any requested script, since the programmer will have already defined the input type and name attributes as part of the schema.
This was written back when Python 2.7 was cool, so I’ll have to update it to replace ConfigParser and StringIO with the modern equivalents.
And for today’s episode of “I don’t know why it’s not working for you, it works on my system!” I was just testing this for the first time in 2 years and the files it created were empty. Looks like when running python on Windows using the io module, I have to explicitly call valid.file.close() for the file to finish writing. I’ll have to test and update the rest of my example files.