I got a requirement to parse the message files in .txt format real time as and when they arrive in incoming windows directory. The directory is in my local Windows Virtual Machine something like D:/MessageFiles/
I wrote a Python script to parse the message files because it's a fixed width file and it parses all the files in the directory and generates the output. Once the files are successfully parsed, it will be moved to archive directory. Now, i would like to make this script run continuously so that it looks for the incoming message files in the directory D:/MessageFiles/ and perform the processing as and when it sees the new files in the path.
Can someone please let me know how to do this?
Alexis Drakopoulos :
There are a few ways to do this, it depends on how fast you need it to archive the files.\n\nIf the frequency is low, for example every hour, you can try to use windows task scheduler to run the python script.\n\nIf we are talking high frequency, or you really want a python script running 24/7, you could put it in a while loop and at the end of the loop do time.sleep()\n\nIf you go with this, I would recommend not blindly parsing the entire directory on every run, but instead finding a way to check whether new files have been added to the directory (such as the amount of files perhaps, or the total size). And then if there is a fluctuation you can archive.",
2018-10-20T13:24:30