2019-06-06
This commit is contained in:
parent
f0de17cd3b
commit
fd91dd1894
34
tasks/DIN-323 Proxies/Note 2019-05-30T13.09.58.md
Normal file
34
tasks/DIN-323 Proxies/Note 2019-05-30T13.09.58.md
Normal file
@ -0,0 +1,34 @@
|
||||
Note 2019-05-30T13.09.58
|
||||
========================
|
||||
|
||||
Evaluate Options to replace Proxy Mesh given procurement review of this supplier:
|
||||
|
||||
Option 1: https://oxylabs.io/
|
||||
|
||||
Option 2: https://luminati.io/
|
||||
|
||||
Conduct a review of each potential supplier and assess best technical and procurement provider
|
||||
|
||||
|
||||
Option 1
|
||||
---
|
||||
* Allows Ip whitelisting
|
||||
|
||||
* You can use unlimited number of threads.
|
||||
|
||||
* It is possible to change the IP on every request.
|
||||
|
||||
|
||||
Prices
|
||||
|
||||
| |Residential | | |
|
||||
|--- |---|---|---| --- |
|
||||
| Basic | 50 GB included | $12 per GB | $600 min. monthly commitment |
|
||||
| Pro | 100 GB included | $9 per GB | $900 min. monthly commitment|
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Option 2
|
||||
---
|
@ -1,4 +1,4 @@
|
||||
Note 2019-05-29T11.34.48
|
||||
DIN-329
|
||||
========================
|
||||
|
||||
As discussed, I've added all the defects linked to the Netherlands scrapper to this ticket:
|
||||
@ -25,11 +25,27 @@ Defect 1
|
||||
---
|
||||
The 'CS' ABN Amro Groenbank B V is indexed but not processed
|
||||
|
||||
```
|
||||
[
|
||||
[
|
||||
"Statutory name",
|
||||
"ABN AMRO Groenbank B.V."
|
||||
],
|
||||
[
|
||||
"Trade name",
|
||||
"ABN AMRO Groenbank B.V."
|
||||
],
|
||||
```
|
||||
|
||||
|
||||
Defect 2
|
||||
---
|
||||
NIBC Bank N.V. has a category with 2 items but one item is logged.
|
||||
|
||||
[
|
||||
"Category",
|
||||
"Emittent effecten CSDB, Bank"
|
||||
]
|
||||
|
||||
|
||||
|
||||
@ -41,4 +57,6 @@ Coliding filenames
|
||||
|
||||
The ID query string value is taken from the href link, and the first 8 characters are extracted as a short hash.
|
||||
|
||||
This short hash is added to the filename when it is created.
|
||||
This short hash is added to the filename when it is created.
|
||||
|
||||
Statutory name checking is now implemented. If the name has ben used before, it will use the statuory name and the trade name, otherwise just the statutory name.
|
45
tasks/DIN-329 NL Fixes/commit.md
Normal file
45
tasks/DIN-329 NL Fixes/commit.md
Normal file
@ -0,0 +1,45 @@
|
||||
DIN-329 - Netherlands NCA Data Defects
|
||||
|
||||
# Summary
|
||||
|
||||
*Defect 1:*
|
||||
|
||||
The menu was using the wrong counter, which gets used later on for processing credit institutes. A new counter was added specifically for building the index.
|
||||
|
||||
```
|
||||
[
|
||||
[
|
||||
"Statutory name",
|
||||
"ABN AMRO Groenbank B.V."
|
||||
],
|
||||
[
|
||||
"Trade name",
|
||||
"ABN AMRO Groenbank B.V."
|
||||
],
|
||||
```
|
||||
|
||||
*Defect 2:*
|
||||
|
||||
Test created and returns the correct value. It could be possible that the page was updated between scraping and testing.
|
||||
|
||||
*Defect 3:*
|
||||
|
||||
The ID query string value is taken from the href link, and the first 8 characters are extracted as a short hash.
|
||||
|
||||
This short hash is added to the filename when it is created.
|
||||
|
||||
Statutory name checking is now implemented. If the name has ben used before, it will use the statuory name and the trade name, otherwise just the statutory name.
|
||||
|
||||
|
||||
# Test Plan
|
||||
|
||||
```
|
||||
|
||||
# NL:: Scrape a Payment Instititute
|
||||
...
|
||||
# NL:: DIN-329 defects
|
||||
# NL::Defect 2
|
||||
ok 25 Extracted Details from Page correctly
|
||||
|
||||
|
||||
```
|
1
tasks/DIN-341 B3 Epic/.~lock.timings.ods#
Normal file
1
tasks/DIN-341 B3 Epic/.~lock.timings.ods#
Normal file
@ -0,0 +1 @@
|
||||
,martin,Cyan-Corvus-414,05.06.2019 15:46,file:///home/martin/.config/libreoffice/4;
|
48
tasks/DIN-341 B3 Epic/Note 2019-05-30T15.24.02.md
Normal file
48
tasks/DIN-341 B3 Epic/Note 2019-05-30T15.24.02.md
Normal file
@ -0,0 +1,48 @@
|
||||
Note 2019-05-30T15.24.02
|
||||
========================
|
||||
|
||||
Spain,
|
||||
Estonia
|
||||
Norway
|
||||
Latvia
|
||||
Denmark
|
||||
|
||||
|
||||
|
||||
"IE_CRON",
|
||||
"NL_CRON",
|
||||
"FR_CRON",
|
||||
"CY_CRON",
|
||||
"SE_CRON",
|
||||
"DE_CRON",
|
||||
"IT_CRON",
|
||||
"LU_CRON",
|
||||
"LT_CRON",
|
||||
"PL_CRON",
|
||||
"SK_CRON",
|
||||
"PT_CRON",
|
||||
|
||||
|
||||
|
||||
|
||||
Ireland, Netherlands, France, Cyprus, Sweden, Germany, Italy, Luxumbourg, Lithuania, Poland, Slovakia, Portugal
|
||||
|
||||
|
||||
|
||||
Spain, Estonia, Norway, Latvia & Denmark
|
||||
|
||||
Deploy Estonia to PPE and test
|
||||
|
||||
|
||||
Activate the entry in the terraform settings, and ensure that the service works correctly with the other services.
|
||||
|
||||
|
||||
https://openbanking.atlassian.net/browse/DIN-342
|
||||
|
||||
https://openbanking.atlassian.net/browse/DIN-343
|
||||
|
||||
https://openbanking.atlassian.net/browse/DIN-344
|
||||
|
||||
https://openbanking.atlassian.net/browse/DIN-345
|
||||
|
||||
https://openbanking.atlassian.net/browse/DIN-346
|
6
tasks/DIN-341 B3 Epic/Note 2019-06-03T09.59.54.md
Normal file
6
tasks/DIN-341 B3 Epic/Note 2019-06-03T09.59.54.md
Normal file
@ -0,0 +1,6 @@
|
||||
Note 2019-06-03T09.59.54
|
||||
========================
|
||||
|
||||
"NL_CRON",
|
||||
"IT_CRON",
|
||||
"PT_CRON",
|
BIN
tasks/DIN-341 B3 Epic/timings.ods
Normal file
BIN
tasks/DIN-341 B3 Epic/timings.ods
Normal file
Binary file not shown.
BIN
tasks/DIN-341 B3 Epic/timings.pdf
Normal file
BIN
tasks/DIN-341 B3 Epic/timings.pdf
Normal file
Binary file not shown.
13
tasks/DIN-364 B4 Epic/Note 2019-06-04T12.21.11.md
Normal file
13
tasks/DIN-364 B4 Epic/Note 2019-06-04T12.21.11.md
Normal file
@ -0,0 +1,13 @@
|
||||
Note 2019-06-04T12.21.11
|
||||
========================
|
||||
|
||||
Belgium
|
||||
Czech
|
||||
Gibralter
|
||||
Malta
|
||||
Croatia
|
||||
|
||||
|
||||
|
||||
Get Belgium, Czech, Gibralter, Malta & Croatia running on PPE.
|
||||
|
19
tasks/DIN-370 B5 Epic/Note 2019-06-04T12.22.24.md
Normal file
19
tasks/DIN-370 B5 Epic/Note 2019-06-04T12.22.24.md
Normal file
@ -0,0 +1,19 @@
|
||||
Note 2019-06-04T12.22.24
|
||||
========================
|
||||
|
||||
Bulgaria
|
||||
Romania
|
||||
Austria
|
||||
Slovenia
|
||||
Greece
|
||||
|
||||
|
||||
|
||||
Get Bulgaria, Romania, Austria, Slovenia & Greece running on PPE.
|
||||
|
||||
Deploy Greece obdfcascrape to PPE and test
|
||||
|
||||
|
||||
There are 5 countries which need to be deploy to PPE.
|
||||
|
||||
Activate the entry in the terraform settings, and ensure that the service works correctly with the other services.
|
@ -0,0 +1 @@
|
||||
,martin,Cyan-Corvus-414,04.06.2019 16:27,file:///home/martin/.config/libreoffice/4;
|
Binary file not shown.
@ -1,4 +1,3 @@
|
||||
Comment
|
||||
Note 2019-05-29T16.34.38
|
||||
========================
|
||||
|
||||
|
@ -1,3 +0,0 @@
|
||||
Note 2019-05-07T09.47.02
|
||||
========================
|
||||
|
Loading…
Reference in New Issue
Block a user