I'm working on replacing our FTP process (mostly because we have to; The old servers are being removed.) But I'm stuck across the board. After configuring everything on the AWS side, we can see into the bucket, see files, but I see several problems on the horizon:
- The file name changes every day. I can't select a specific file name unless we change how files are processed.
- The file doesn't get removed from the S3 bucket, so there's no way for the original team to simply replace the same named file (if they're even willing to do that.)
- The AWS pull seems to demand that I basically take the excel sheet and put it onto a record, rather than importing it into a staging table that will then transform the sheet and process the data.
This is a heavily important process, as it handles all of our onboarding and offboarding, and there doesn't seem to be any smooth transitions. Any assistance would be greatly appreciated.
