592 B
592 B
DIR-3159 Extend OBDFCASCRAPE to download Malta NCA data
Summary
-
Added archiving - The contents of the artefact folder is compressed
-
Added an index of the pages scraped to aid ingestion
-
Changed filenames to ensure there are no collisions, items are prefixed with either ps_, em or ci_.
-
Implemented Amazon SQS message service so that an announcement can be made when a new file is avilable for ingestion
-
Added dictioary loader and auto saver to the main Scraper object.
-
Tests for the scraper
-
Added the correct whois service for Malta
-
Implemented auto mated translate