conda: Yield listed origins after all artifacts in a page are processed

swh-scheduler will deduplicate listed origins according to their URL and visit type but not according to their extra loader arguments.

Previously, listed origins were yielded after each processed artifact in a page so we could lose some package version info due to the deduplication process.

So ensure to yield listed origins once all artifacts in a page have been processed.

---

Migrated from D8747 (view on Phabricator)

Build is green

Patch application report for D8747 (id=31527)

Rebasing onto 0baaf68c...

Current branch diff-target is up to date.

Changes applied before test

commit f001bc1bc1695af7d21cd8fad33d00ded51ccfcd
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Oct 19 15:59:25 2022 +0200

    conda: Yield listed origins after all artifacts in a page processed
    
    swh-scheduler will deduplicate listed origins according to their URL
    and visit type but not according to their extra loader arguments.
    
    Previously, listed origins were yielded after each processed artifact
    in a page so we could lose some package version info due to the
    deduplication process.
    
    So ensure to yield listed origins once all artifacts in a page have
    been processed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/803/ for more details.

I believe the commit title lacks a verb, doesn't it? ("conda: Yield listed origins after all artifacts in a page are processed" or something similar?)

shouldn't this fix come with a test of some sort?

Add missing test

Build is green

Patch application report for D8747 (id=31531)

Rebasing onto 0baaf68c...

Current branch diff-target is up to date.

Changes applied before test

commit fc22f7539a7fda5294ae8135d7c08d86b4ad0870
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Oct 19 15:59:25 2022 +0200

    conda: Yield listed origins after all artifacts in a page are processed
    
    swh-scheduler will deduplicate listed origins according to their URL
    and visit type but not according to their extra loader arguments.
    
    Previously, listed origins were yielded after each processed artifact
    in a page so we could lose some package version info due to the
    deduplication process.
    
    So ensure to yield listed origins once all artifacts in a page have
    been processed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/805/ for more details.

Could you add docstrings to the test? I don't understand what they are expected to test

Merge request was returned for changes

Rebase and address @vlorentz comments

Build is green

Patch application report for D8747 (id=31568)

Rebasing onto 8a82bbf9...

Current branch diff-target is up to date.

Changes applied before test

commit 018fc641bfb2c578be762f1867c33101687bfc03
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Oct 19 15:59:25 2022 +0200

    conda: Yield listed origins after all artifacts in a page are processed
    
    swh-scheduler will deduplicate listed origins according to their URL
    and visit type but not according to their extra loader arguments.
    
    Previously, listed origins were yielded after each processed artifact
    in a page so we could lose some package version info due to the
    deduplication process.
    
    So ensure to yield listed origins once all artifacts in a page have
    been processed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/811/ for more details.

perfect, thanks

Merge request was accepted

approved this merge request

Rebase

Build is green

Patch application report for D8747 (id=31592)

Rebasing onto 31eb5f63...

Current branch diff-target is up to date.

Changes applied before test

commit 4f6b3f3f09b8a8ab8a5e3679b3a8b936dbd21640
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Oct 19 15:59:25 2022 +0200

    conda: Yield listed origins after all artifacts in a page are processed
    
    swh-scheduler will deduplicate listed origins according to their URL
    and visit type but not according to their extra loader arguments.
    
    Previously, listed origins were yielded after each processed artifact
    in a page so we could lose some package version info due to the
    deduplication process.
    
    So ensure to yield listed origins once all artifacts in a page have
    been processed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/823/ for more details.

Merge request was merged

closed

https://gitlab.softwareheritage.org/swh/meta/-/issues/4546