Use beautifulsoup4 CSS selectors to simplify code and type checking
As the types-beautifulsoup4
package gets installed in the swh virtualenv
as it is a swh-scanner
test dependency, some mypy errors were reported
related to beautifulsoup4
typing.
As the returned type for the find
method of bs4 is the following union:
Tag | NavigableString | None
, isinstance
calls must be used to ensure
proper typing which is not great.
So prefer to use the select_one
method instead where a simple None
check
must be done to ensure typing is correct as it is returning Optional[Tag]
.
In a similar manner, replace use of find_all
method by select
method.
It also has the advantage to simplify the code.