cc-tracking/tasks/DIR-3111/commit.md

11 lines
428 B
Markdown
Raw Normal View History

2019-03-21 11:11:07 +00:00
DIR-3111 Extend OBDFCASCRAPE to be able to upload Swedish NCA data
# Summary
* Added archiving - The contents of the artefact folder is compressed
* Added an index of the pages scraped to aid ingestion
* Changed filenames to ensure there are no collisions, items are prefixed with either ps_, em or ci_.
* Implemented Amazon SQS message service so that an announcement can be made when a new file is avilable for ingestion