Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] SSLCertVerificationError for URL with valid certificate. #310

Closed
vishalnandagopal opened this issue Jun 1, 2022 · 6 comments
Closed

Comments

@vishalnandagopal
Copy link

When I try to fetch https://lobste.rs/t/security.rss, it shows a SSLCertVerificationError even though the site is opening for me on Firefox, Chrome, the terminal (through curl) and I am also able to fetch the feed through the requests library (without disabling certificate verification).

Do you have any idea why an SSL error occurring on feedparser? Does it use a custom certificate library? The site has a valid certificate and I also ran tests from SSL Labs on it (It got an A+ grade).

>>> import feedparser                                                             
>>> feed = feedparser.parse("https://lobste.rs/t/security.rss")                   
>>> print(feed)
{'bozo': True, 'entries': [], 'feed': {}, 'headers': {}, 'bozo_exception': URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)'))}

For now, I am fetching the feed through requests.get(url) and then sending it to feedparser, but I am concerned about the security implications of sending untrusted xml/html to feedparser.

>>> import requests, feedparser
>>> html_text = requests.get("https://lobste.rs/t/security.rss").text
>>> feedparser.parse(html_text)
{'bozo': False, 'entries': [{'title': 'Curated list of KeePass-related projects', 'title_detail': {'type': 'text/plain', 'language': None, 'base': '', 'value': 'Curated list of KeePass-related projects'}, 'links': [{ .....
@vishalnandagopal
Copy link
Author

Related to #84 #290 #168

@n3s7or
Copy link

n3s7or commented Jun 13, 2022

I just tested this and in my environment it worked fine:

Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import feedparser
>>> url = 'https://lobste.rs/t/security.rss'
>>> feedparser.parse(url)
{'bozo': False, 'entries': [{'title': 'Making popular Ruby packages more secure', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Making popular Ruby packages more secure'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://blog.rubygems.org/2022/06/13/making-packages-more-secure.html'}], 'link': 'https://blog.rubygems.org/2022/06/13/making-packages-more-secure.html', 'id': 'https://lobste.rs/s/vduzkn', 'guidislink': False, 'authors': [{'name': 'telemachus', 'email': 'telemachus@users.lobste.rs'}], 'author': 'telemachus@users.lobste.rs (telemachus)', 'author_detail': {'name': 'telemachus', 'email': 'telemachus@users.lobste.rs'}, 'published': 'Mon, 13 Jun 2022 16:03:17 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=13, tm_hour=21, tm_min=3, tm_sec=17, tm_wday=0, tm_yday=164, tm_isdst=0), 'comments': 'https://lobste.rs/s/vduzkn/making_popular_ruby_packages_more_secure', 'summary': '<p><a href="https://lobste.rs/s/vduzkn/making_popular_ruby_packages_more_secure">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/vduzkn/making_popular_ruby_packages_more_secure">Comments</a></p>'}, 'tags': [{'term': 'ruby', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Symbiote: A New, Nearly-Impossible-to-Detect Linux Threat', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Symbiote: A New, Nearly-Impossible-to-Detect Linux Threat'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://blogs.blackberry.com/en/2022/06/symbiote-a-new-nearly-impossible-to-detect-linux-threat'}], 'link': 'https://blogs.blackberry.com/en/2022/06/symbiote-a-new-nearly-impossible-to-detect-linux-threat', 'id': 'https://lobste.rs/s/ijottt', 'guidislink': False, 'authors': [{'name': 'Yogthos', 'email': 'Yogthos@users.lobste.rs'}], 'author': 'Yogthos@users.lobste.rs (Yogthos)', 'author_detail': {'name': 'Yogthos', 'email': 'Yogthos@users.lobste.rs'}, 'published': 'Fri, 10 Jun 2022 13:46:46 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=10, tm_hour=18, tm_min=46, tm_sec=46, tm_wday=4, tm_yday=161, tm_isdst=0), 'comments': 'https://lobste.rs/s/ijottt/symbiote_new_nearly_impossible_detect', 'summary': '<p><a href="https://lobste.rs/s/ijottt/symbiote_new_nearly_impossible_detect">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/ijottt/symbiote_new_nearly_impossible_detect">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'linux', 'scheme': None, 'label': None}]}, {'title': 'If OpenSSL were a GUI', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'If OpenSSL were a GUI'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://smallstep.com/blog/if-openssl-were-a-gui/'}], 'link': 'https://smallstep.com/blog/if-openssl-were-a-gui/', 'id': 'https://lobste.rs/s/faymlu', 'guidislink': False, 'authors': [{'name': 'friendlysock', 'email': 'friendlysock@users.lobste.rs'}], 'author': 'friendlysock@users.lobste.rs (friendlysock)', 'author_detail': {'name': 'friendlysock', 'email': 'friendlysock@users.lobste.rs'}, 'published': 'Fri, 10 Jun 2022 13:30:22 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=10, tm_hour=18, tm_min=30, tm_sec=22, tm_wday=4, tm_yday=161, tm_isdst=0), 'comments': 'https://lobste.rs/s/faymlu/if_openssl_were_gui', 'summary': '<p><a href="https://lobste.rs/s/faymlu/if_openssl_were_gui">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/faymlu/if_openssl_were_gui">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'satire', 'scheme': None, 'label': None}]}, {'title': 'PACMAN: a novel hardware attack that can bypass Pointer Authentication (PAC) on the Apple M1 CPU', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'PACMAN: a novel hardware attack that can bypass Pointer Authentication (PAC) on the Apple M1 CPU'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://pacmanattack.com/'}], 'link': 'https://pacmanattack.com/', 'id': 'https://lobste.rs/s/dxhkyn', 'guidislink': False, 'authors': [{'name': 'lattera', 'email': 'lattera@users.lobste.rs'}], 'author': 'lattera@users.lobste.rs (lattera)', 'author_detail': {'name': 'lattera', 'email': 'lattera@users.lobste.rs'}, 'published': 'Fri, 10 Jun 2022 12:10:39 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=10, tm_hour=17, tm_min=10, tm_sec=39, tm_wday=4, tm_yday=161, tm_isdst=0), 'comments': 'https://lobste.rs/s/dxhkyn/pacman_novel_hardware_attack_can_bypass', 'summary': '<p><a href="https://lobste.rs/s/dxhkyn/pacman_novel_hardware_attack_can_bypass">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/dxhkyn/pacman_novel_hardware_attack_can_bypass">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Private Access Tokens: eliminating CAPTCHAs on iPhones and Macs with open standards', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Private Access Tokens: eliminating CAPTCHAs on iPhones and Macs with open standards'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/'}], 'link': 'https://blog.cloudflare.com/eliminating-captchas-on-iphones-and-macs-using-new-standard/', 'id': 'https://lobste.rs/s/2xzc7k', 'guidislink': False, 'authors': [{'name': 'adamshaylor', 'email': 'adamshaylor@users.lobste.rs'}], 'author': 'adamshaylor@users.lobste.rs (adamshaylor)', 'author_detail': {'name': 'adamshaylor', 'email': 'adamshaylor@users.lobste.rs'}, 'published': 'Wed, 08 Jun 2022 22:25:35 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=9, tm_hour=3, tm_min=25, tm_sec=35, tm_wday=3, tm_yday=160, tm_isdst=0), 'comments': 'https://lobste.rs/s/2xzc7k/private_access_tokens_eliminating', 'summary': '<p><a href="https://lobste.rs/s/2xzc7k/private_access_tokens_eliminating">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/2xzc7k/private_access_tokens_eliminating">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'release', 'scheme': None, 'label': None}, {'term': 'web', 'scheme': None, 'label': None}]}, {'title': 'Encrypted Databases: From Theory to Systems', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Encrypted Databases: From Theory to Systems'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.cidrdb.org/cidr2021/papers/cidr2021_paper23.pdf'}], 'link': 'https://www.cidrdb.org/cidr2021/papers/cidr2021_paper23.pdf', 'id': 'https://lobste.rs/s/iunobp', 'guidislink': False, 'authors': [{'name': 'df', 'email': 'df@users.lobste.rs'}], 'author': 'df@users.lobste.rs (df)', 'author_detail': {'name': 'df', 'email': 'df@users.lobste.rs'}, 'published': 'Wed, 08 Jun 2022 12:08:21 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=8, tm_hour=17, tm_min=8, tm_sec=21, tm_wday=2, tm_yday=159, tm_isdst=0), 'comments': 'https://lobste.rs/s/iunobp/encrypted_databases_from_theory_systems', 'summary': '<p><a href="https://lobste.rs/s/iunobp/encrypted_databases_from_theory_systems">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/iunobp/encrypted_databases_from_theory_systems">Comments</a></p>'}, 'tags': [{'term': 'pdf', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'databases', 'scheme': None, 'label': None}, {'term': 'cryptography', 'scheme': None, 'label': None}]}, {'title': 'SFUZZ: High Performance Coverage-guided Greybox Fuzzer with Custom JIT Engine', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'SFUZZ: High Performance Coverage-guided Greybox Fuzzer with Custom JIT Engine'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://seal9055.com/blog/?p=sfuzz&d=fuzzing'}], 'link': 'https://seal9055.com/blog/?p=sfuzz&d=fuzzing', 'id': 'https://lobste.rs/s/anwmfb', 'guidislink': False, 'authors': [{'name': 'jakob', 'email': 'jakob@users.lobste.rs'}], 'author': 'jakob@users.lobste.rs (jakob)', 'author_detail': {'name': 'jakob', 'email': 'jakob@users.lobste.rs'}, 'published': 'Wed, 08 Jun 2022 06:27:23 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=8, tm_hour=11, tm_min=27, tm_sec=23, tm_wday=2, tm_yday=159, tm_isdst=0), 'comments': 'https://lobste.rs/s/anwmfb/sfuzz_high_performance_coverage_guided', 'summary': '<p><a href="https://lobste.rs/s/anwmfb/sfuzz_high_performance_coverage_guided">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/anwmfb/sfuzz_high_performance_coverage_guided">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'assembly', 'scheme': None, 'label': None}]}, {'title': 'dot: The Deepfake Offensive Toolkit', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'dot: The Deepfake Offensive Toolkit'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://github.com/sensity-ai/dot'}], 'link': 'https://github.com/sensity-ai/dot', 'id': 'https://lobste.rs/s/uuhty9', 'guidislink': False, 'authors': [{'name': 'feoh', 'email': 'feoh@users.lobste.rs'}], 'author': 'feoh@users.lobste.rs (feoh)', 'author_detail': {'name': 'feoh', 'email': 'feoh@users.lobste.rs'}, 'published': 'Tue, 07 Jun 2022 09:19:35 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=7, tm_hour=14, tm_min=19, tm_sec=35, tm_wday=1, tm_yday=158, tm_isdst=0), 'comments': 'https://lobste.rs/s/uuhty9/dot_deepfake_offensive_toolkit', 'summary': '<p><a href="https://lobste.rs/s/uuhty9/dot_deepfake_offensive_toolkit">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/uuhty9/dot_deepfake_offensive_toolkit">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'ai', 'scheme': None, 'label': None}]}, {'title': 'A layered approach to content blocking', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'A layered approach to content blocking'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://seirdy.one/posts/2022/06/04/layered-content-blocking/'}], 'link': 'https://seirdy.one/posts/2022/06/04/layered-content-blocking/', 'id': 'https://lobste.rs/s/zg8ssb', 'guidislink': False, 'authors': [{'name': 'Seirdy', 'email': 'Seirdy@users.lobste.rs'}], 'author': 'Seirdy@users.lobste.rs (Seirdy)', 'author_detail': {'name': 'Seirdy', 'email': 'Seirdy@users.lobste.rs'}, 'published': 'Mon, 06 Jun 2022 19:09:58 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=7, tm_hour=0, tm_min=9, tm_sec=58, tm_wday=1, tm_yday=158, tm_isdst=0), 'comments': 'https://lobste.rs/s/zg8ssb/layered_approach_content_blocking', 'summary': '<p><a href="https://lobste.rs/s/zg8ssb/layered_approach_content_blocking">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/zg8ssb/layered_approach_content_blocking">Comments</a></p>'}, 'tags': [{'term': 'browsers', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'a11y', 'scheme': None, 'label': None}]}, {'title': "Exploiting the Wii U's USB Descriptor parsing", 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': "Exploiting the Wii U's USB Descriptor parsing"}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://garyodernichts.blogspot.com/2022/06/exploiting-wii-us-usb-descriptor-parsing.html'}], 'link': 'https://garyodernichts.blogspot.com/2022/06/exploiting-wii-us-usb-descriptor-parsing.html', 'id': 'https://lobste.rs/s/e3q2mo', 'guidislink': False, 'authors': [{'name': 'crazyloglad', 'email': 'crazyloglad@users.lobste.rs'}], 'author': 'crazyloglad@users.lobste.rs (crazyloglad)', 'author_detail': {'name': 'crazyloglad', 'email': 'crazyloglad@users.lobste.rs'}, 'published': 'Mon, 06 Jun 2022 14:52:46 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=6, tm_hour=19, tm_min=52, tm_sec=46, tm_wday=0, tm_yday=157, tm_isdst=0), 'comments': 'https://lobste.rs/s/e3q2mo/exploiting_wii_u_s_usb_descriptor_parsing', 'summary': '<p><a href="https://lobste.rs/s/e3q2mo/exploiting_wii_u_s_usb_descriptor_parsing">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/e3q2mo/exploiting_wii_u_s_usb_descriptor_parsing">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Deep inside TOMOYO Linux (2009)', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Deep inside TOMOYO Linux (2009)'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://osdn.net/projects/tomoyo/docs/lca2009-kumaneko.pdf'}], 'link': 'https://osdn.net/projects/tomoyo/docs/lca2009-kumaneko.pdf', 'id': 'https://lobste.rs/s/g26kdf', 'guidislink': False, 'authors': [{'name': 'calvin', 'email': 'calvin@users.lobste.rs'}], 'author': 'calvin@users.lobste.rs (calvin)', 'author_detail': {'name': 'calvin', 'email': 'calvin@users.lobste.rs'}, 'published': 'Sun, 05 Jun 2022 11:12:23 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=5, tm_hour=16, tm_min=12, tm_sec=23, tm_wday=6, tm_yday=156, tm_isdst=0), 'comments': 'https://lobste.rs/s/g26kdf/deep_inside_tomoyo_linux_2009', 'summary': '<p><a href="https://lobste.rs/s/g26kdf/deep_inside_tomoyo_linux_2009">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/g26kdf/deep_inside_tomoyo_linux_2009">Comments</a></p>'}, 'tags': [{'term': 'pdf', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'linux', 'scheme': None, 'label': None}]}, {'title': 'An Apple a day keeps the exploiter away', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'An Apple a day keeps the exploiter away'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.sstic.org/media/SSTIC2022/SSTIC-actes/an_apple_a_day/SSTIC2022-Slides-an_apple_a_day-benoist-vanderbeken_perigaud.pdf'}], 'link': 'https://www.sstic.org/media/SSTIC2022/SSTIC-actes/an_apple_a_day/SSTIC2022-Slides-an_apple_a_day-benoist-vanderbeken_perigaud.pdf', 'id': 'https://lobste.rs/s/pc0ssd', 'guidislink': False, 'authors': [{'name': 'fro', 'email': 'fro@users.lobste.rs'}], 'author': 'fro@users.lobste.rs (fro)', 'author_detail': {'name': 'fro', 'email': 'fro@users.lobste.rs'}, 'published': 'Sun, 05 Jun 2022 11:11:43 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=5, tm_hour=16, tm_min=11, tm_sec=43, tm_wday=6, tm_yday=156, tm_isdst=0), 'comments': 'https://lobste.rs/s/pc0ssd/apple_day_keeps_exploiter_away', 'summary': '<p><a href="https://lobste.rs/s/pc0ssd/apple_day_keeps_exploiter_away">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/pc0ssd/apple_day_keeps_exploiter_away">Comments</a></p>'}, 'tags': [{'term': 'pdf', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'ios', 'scheme': None, 'label': None}, {'term': 'slides', 'scheme': None, 'label': None}]}, {'title': 'Boot drive encryption security on Windows', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Boot drive encryption security on Windows'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://threedots.ovh/blog/2022/06/boot-drive-encryption-security-on-windows/'}], 'link': 'https://threedots.ovh/blog/2022/06/boot-drive-encryption-security-on-windows/', 'id': 'https://lobste.rs/s/ikzqnw', 'guidislink': False, 'authors': [{'name': 'never_released', 'email': 'never_released@users.lobste.rs'}], 'author': 'never_released@users.lobste.rs (never_released)', 'author_detail': {'name': 'never_released', 'email': 'never_released@users.lobste.rs'}, 'published': 'Sat, 04 Jun 2022 09:31:40 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=4, tm_hour=14, tm_min=31, tm_sec=40, tm_wday=5, tm_yday=155, tm_isdst=0), 'comments': 'https://lobste.rs/s/ikzqnw/boot_drive_encryption_security_on', 'summary': '<p><a href="https://lobste.rs/s/ikzqnw/boot_drive_encryption_security_on">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/ikzqnw/boot_drive_encryption_security_on">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'windows', 'scheme': None, 'label': None}]}, {'title': 'Apple Silicon “Augury” DMP Vulnerability', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Apple Silicon “Augury” DMP Vulnerability'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://mjtsai.com/blog/2022/06/03/apple-silicon-augury-dmp-vulnerability/'}], 'link': 'https://mjtsai.com/blog/2022/06/03/apple-silicon-augury-dmp-vulnerability/', 'id': 'https://lobste.rs/s/rqpqje', 'guidislink': False, 'authors': [{'name': 'Gaelan', 'email': 'Gaelan@users.lobste.rs'}], 'author': 'Gaelan@users.lobste.rs (Gaelan)', 'author_detail': {'name': 'Gaelan', 'email': 'Gaelan@users.lobste.rs'}, 'published': 'Fri, 03 Jun 2022 11:43:43 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=3, tm_hour=16, tm_min=43, tm_sec=43, tm_wday=4, tm_yday=154, tm_isdst=0), 'comments': 'https://lobste.rs/s/rqpqje/apple_silicon_augury_dmp_vulnerability', 'summary': '<p><a href="https://lobste.rs/s/rqpqje/apple_silicon_augury_dmp_vulnerability">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/rqpqje/apple_silicon_augury_dmp_vulnerability">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'hardware', 'scheme': None, 'label': None}]}, {'title': 'Dangerous Gift', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Dangerous Gift'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.tbray.org/ongoing/When/202x/2022/06/02/Dangerous-Gift'}], 'link': 'https://www.tbray.org/ongoing/When/202x/2022/06/02/Dangerous-Gift', 'id': 'https://lobste.rs/s/zemzol', 'guidislink': False, 'authors': [{'name': 'napkindrawing', 'email': 'napkindrawing@users.lobste.rs'}], 'author': 'napkindrawing@users.lobste.rs (napkindrawing)', 'author_detail': {'name': 'napkindrawing', 'email': 'napkindrawing@users.lobste.rs'}, 'published': 'Fri, 03 Jun 2022 08:42:36 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=3, tm_hour=13, tm_min=42, tm_sec=36, tm_wday=4, tm_yday=154, tm_isdst=0), 'comments': 'https://lobste.rs/s/zemzol/dangerous_gift', 'summary': '<p><a href="https://lobste.rs/s/zemzol/dangerous_gift">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/zemzol/dangerous_gift">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'law', 'scheme': None, 'label': None}]}, {'title': 'Zero-Day Exploitation of Atlassian Confluence', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Zero-Day Exploitation of Atlassian Confluence'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.volexity.com/blog/2022/06/02/zero-day-exploitation-of-atlassian-confluence/'}], 'link': 'https://www.volexity.com/blog/2022/06/02/zero-day-exploitation-of-atlassian-confluence/', 'id': 'https://lobste.rs/s/fhsz9q', 'guidislink': False, 'authors': [{'name': 'dijit', 'email': 'dijit@users.lobste.rs'}], 'author': 'dijit@users.lobste.rs (dijit)', 'author_detail': {'name': 'dijit', 'email': 'dijit@users.lobste.rs'}, 'published': 'Thu, 02 Jun 2022 23:17:06 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=3, tm_hour=4, tm_min=17, tm_sec=6, tm_wday=4, tm_yday=154, tm_isdst=0), 'comments': 'https://lobste.rs/s/fhsz9q/zero_day_exploitation_atlassian', 'summary': '<p><a href="https://lobste.rs/s/fhsz9q/zero_day_exploitation_atlassian">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/fhsz9q/zero_day_exploitation_atlassian">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Use compiler flags for stack protection in GCC and Clang', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Use compiler flags for stack protection in GCC and Clang'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://developers.redhat.com/articles/2022/06/02/use-compiler-flags-stack-protection-gcc-and-clang'}], 'link': 'https://developers.redhat.com/articles/2022/06/02/use-compiler-flags-stack-protection-gcc-and-clang', 'id': 'https://lobste.rs/s/malqbu', 'guidislink': False, 'authors': [{'name': 'fcambus', 'email': 'fcambus@users.lobste.rs'}], 'author': 'fcambus@users.lobste.rs (fcambus)', 'author_detail': {'name': 'fcambus', 'email': 'fcambus@users.lobste.rs'}, 'published': 'Thu, 02 Jun 2022 16:35:24 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=2, tm_hour=21, tm_min=35, tm_sec=24, tm_wday=3, tm_yday=153, tm_isdst=0), 'comments': 'https://lobste.rs/s/malqbu/use_compiler_flags_for_stack_protection', 'summary': '<p><a href="https://lobste.rs/s/malqbu/use_compiler_flags_for_stack_protection">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/malqbu/use_compiler_flags_for_stack_protection">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'compilers', 'scheme': None, 'label': None}]}, {'title': 'Curated list of KeePass-related projects', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Curated list of KeePass-related projects'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://github.com/lgg/awesome-keepass'}], 'link': 'https://github.com/lgg/awesome-keepass', 'id': 'https://lobste.rs/s/vkcw9u', 'guidislink': False, 'authors': [{'name': 'atoponce', 'email': 'atoponce@users.lobste.rs'}], 'author': 'atoponce@users.lobste.rs (atoponce)', 'author_detail': {'name': 'atoponce', 'email': 'atoponce@users.lobste.rs'}, 'published': 'Tue, 31 May 2022 16:40:17 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=31, tm_hour=21, tm_min=40, tm_sec=17, tm_wday=1, tm_yday=151, tm_isdst=0), 'comments': 'https://lobste.rs/s/vkcw9u/curated_list_keepass_related_projects', 'summary': '<p><a href="https://lobste.rs/s/vkcw9u/curated_list_keepass_related_projects">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/vkcw9u/curated_list_keepass_related_projects">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Microsoft Office RCE - “Follina” MSDT Attack', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Microsoft Office RCE - “Follina” MSDT Attack'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.huntress.com/blog/microsoft-office-remote-code-execution-follina-msdt-bug'}], 'link': 'https://www.huntress.com/blog/microsoft-office-remote-code-execution-follina-msdt-bug', 'id': 'https://lobste.rs/s/twpfqm', 'guidislink': False, 'authors': [{'name': 'JulianWgs', 'email': 'JulianWgs@users.lobste.rs'}], 'author': 'JulianWgs@users.lobste.rs (JulianWgs)', 'author_detail': {'name': 'JulianWgs', 'email': 'JulianWgs@users.lobste.rs'}, 'published': 'Tue, 31 May 2022 13:17:37 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=31, tm_hour=18, tm_min=17, tm_sec=37, tm_wday=1, tm_yday=151, tm_isdst=0), 'comments': 'https://lobste.rs/s/twpfqm/microsoft_office_rce_follina_msdt_attack', 'summary': '<p><a href="https://lobste.rs/s/twpfqm/microsoft_office_rce_follina_msdt_attack">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/twpfqm/microsoft_office_rce_follina_msdt_attack">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Certificate Transparency data is used to compromise WordPress before installation', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Certificate Transparency data is used to compromise WordPress before installation'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.feistyduck.com/bulletproof-tls-newsletter/issue_89_certificate_transparency_data_is_used_to_compromise_wordpress_before_installation'}], 'link': 'https://www.feistyduck.com/bulletproof-tls-newsletter/issue_89_certificate_transparency_data_is_used_to_compromise_wordpress_before_installation', 'id': 'https://lobste.rs/s/hb5tdl', 'guidislink': False, 'authors': [{'name': 'hanno', 'email': 'hanno@users.lobste.rs'}], 'author': 'hanno@users.lobste.rs (hanno)', 'author_detail': {'name': 'hanno', 'email': 'hanno@users.lobste.rs'}, 'published': 'Tue, 31 May 2022 07:19:21 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=31, tm_hour=12, tm_min=19, tm_sec=21, tm_wday=1, tm_yday=151, tm_isdst=0), 'comments': 'https://lobste.rs/s/hb5tdl/certificate_transparency_data_is_used', 'summary': '<p><a href="https://lobste.rs/s/hb5tdl/certificate_transparency_data_is_used">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/hb5tdl/certificate_transparency_data_is_used">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}, {'term': 'web', 'scheme': None, 'label': None}, {'term': 'cryptography', 'scheme': None, 'label': None}]}, {'title': "Using Verizon's API to get a vanity phone number with 4 consecutive digits", 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': "Using Verizon's API to get a vanity phone number with 4 consecutive digits"}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://blog.jonlu.ca/posts/verizon-rare-numbers?ref=ls'}], 'link': 'https://blog.jonlu.ca/posts/verizon-rare-numbers?ref=ls', 'id': 'https://lobste.rs/s/ivakz5', 'guidislink': False, 'authors': [{'name': 'JonLuca', 'email': 'JonLuca@users.lobste.rs'}], 'author': 'JonLuca@users.lobste.rs (JonLuca)', 'author_detail': {'name': 'JonLuca', 'email': 'JonLuca@users.lobste.rs'}, 'published': 'Mon, 30 May 2022 09:26:24 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=30, tm_hour=14, tm_min=26, tm_sec=24, tm_wday=0, tm_yday=150, tm_isdst=0), 'comments': 'https://lobste.rs/s/ivakz5/using_verizon_s_api_get_vanity_phone', 'summary': '<p><a href="https://lobste.rs/s/ivakz5/using_verizon_s_api_get_vanity_phone">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/ivakz5/using_verizon_s_api_get_vanity_phone">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Set Up HTTPS by Default in Your Browser', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Set Up HTTPS by Default in Your Browser'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.eff.org/https-everywhere/set-https-default-your-browser'}], 'link': 'https://www.eff.org/https-everywhere/set-https-default-your-browser', 'id': 'https://lobste.rs/s/splkqh', 'guidislink': False, 'authors': [{'name': 'jparise', 'email': 'jparise@users.lobste.rs'}], 'author': 'jparise@users.lobste.rs (jparise)', 'author_detail': {'name': 'jparise', 'email': 'jparise@users.lobste.rs'}, 'published': 'Sun, 29 May 2022 20:32:08 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=30, tm_hour=1, tm_min=32, tm_sec=8, tm_wday=0, tm_yday=150, tm_isdst=0), 'comments': 'https://lobste.rs/s/splkqh/set_up_https_by_default_your_browser', 'summary': '<p><a href="https://lobste.rs/s/splkqh/set_up_https_by_default_your_browser">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/splkqh/set_up_https_by_default_your_browser">Comments</a></p>'}, 'tags': [{'term': 'browsers', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'privacy', 'scheme': None, 'label': None}]}, {'title': 'ssenv: Unset all known api token environment variables', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'ssenv: Unset all known api token environment variables'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://github.com/abbbi/ssenv'}], 'link': 'https://github.com/abbbi/ssenv', 'id': 'https://lobste.rs/s/8gx4aw', 'guidislink': False, 'authors': [{'name': 'abi', 'email': 'abi@users.lobste.rs'}], 'author': 'abi@users.lobste.rs (abi)', 'author_detail': {'name': 'abi', 'email': 'abi@users.lobste.rs'}, 'published': 'Sat, 28 May 2022 17:34:35 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=28, tm_hour=22, tm_min=34, tm_sec=35, tm_wday=5, tm_yday=148, tm_isdst=0), 'comments': 'https://lobste.rs/s/8gx4aw/ssenv_unset_all_known_api_token', 'summary': '<p><a href="https://lobste.rs/s/8gx4aw/ssenv_unset_all_known_api_token">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/8gx4aw/ssenv_unset_all_known_api_token">Comments</a></p>'}, 'tags': [{'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'PACMAN: Attacking ARM Pointer Authentication with Speculative Execution', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'PACMAN: Attacking ARM Pointer Authentication with Speculative Execution'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://people.csail.mit.edu/weontaek/pubs/PACMAN_ISCA22.pdf'}], 'link': 'https://people.csail.mit.edu/weontaek/pubs/PACMAN_ISCA22.pdf', 'id': 'https://lobste.rs/s/0l57b2', 'guidislink': False, 'authors': [{'name': 'lattera', 'email': 'lattera@users.lobste.rs'}], 'author': 'lattera@users.lobste.rs (lattera)', 'author_detail': {'name': 'lattera', 'email': 'lattera@users.lobste.rs'}, 'published': 'Sat, 28 May 2022 08:02:38 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=28, tm_hour=13, tm_min=2, tm_sec=38, tm_wday=5, tm_yday=148, tm_isdst=0), 'comments': 'https://lobste.rs/s/0l57b2/pacman_attacking_arm_pointer', 'summary': '<p><a href="https://lobste.rs/s/0l57b2/pacman_attacking_arm_pointer">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/0l57b2/pacman_attacking_arm_pointer">Comments</a></p>'}, 'tags': [{'term': 'pdf', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}]}, {'title': 'Ghost in the Shellcode', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Ghost in the Shellcode'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.notcheckmark.com/2022/05/ghost-in-the-shellcode/'}], 'link': 'https://www.notcheckmark.com/2022/05/ghost-in-the-shellcode/', 'id': 'https://lobste.rs/s/ylrjgq', 'guidislink': False, 'authors': [{'name': 'withzombies', 'email': 'withzombies@users.lobste.rs'}], 'author': 'withzombies@users.lobste.rs (withzombies)', 'author_detail': {'name': 'withzombies', 'email': 'withzombies@users.lobste.rs'}, 'published': 'Fri, 27 May 2022 14:26:17 -0500', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=5, tm_mday=27, tm_hour=19, tm_min=26, tm_sec=17, tm_wday=4, tm_yday=147, tm_isdst=0), 'comments': 'https://lobste.rs/s/ylrjgq/ghost_shellcode', 'summary': '<p><a href="https://lobste.rs/s/ylrjgq/ghost_shellcode">Comments</a></p>', 'summary_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': '<p><a href="https://lobste.rs/s/ylrjgq/ghost_shellcode">Comments</a></p>'}, 'tags': [{'term': 'event', 'scheme': None, 'label': None}, {'term': 'security', 'scheme': None, 'label': None}, {'term': 'historical', 'scheme': None, 'label': None}]}], 'feed': {'title': 'Lobsters: security - Netsec, appsec, and infosec', 'title_detail': {'type': 'text/plain', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'Lobsters: security - Netsec, appsec, and infosec'}, 'subtitle': 'security - Netsec, appsec, and infosec', 'subtitle_detail': {'type': 'text/html', 'language': None, 'base': 'https://lobste.rs/t/security.rss', 'value': 'security - Netsec, appsec, and infosec'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://lobste.rs/'}, {'href': 'https://lobste.rs/t/security.rss', 'rel': 'self', 'type': 'application/rss+xml'}], 'link': 'https://lobste.rs/'}, 'headers': {'server': 'nginx', 'date': 'Mon, 13 Jun 2022 22:47:56 GMT', 'content-type': 'application/rss+xml; charset=utf-8', 'transfer-encoding': 'chunked', 'connection': 'close', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '1; mode=block', 'x-content-type-options': 'nosniff', 'x-download-options': 'noopen', 'x-permitted-cross-domain-policies': 'none', 'referrer-policy': 'strict-origin-when-cross-origin', 'etag': 'W/"4ecfeb39211c45f339a438a1268c0879"', 'cache-control': 'max-age=0, private, must-revalidate', 'content-security-policy-report-only': "default-src 'none'; connect-src 'self'; font-src 'self' https: data:; img-src * data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; form-action 'self'; report-uri /csp-violation-report", 'x-request-id': 'ca1d78d0-7028-4a05-82fd-0b645aa50ecf', 'x-runtime': '0.068228', 'strict-transport-security': 'max-age=63072000; includeSubDomains'}, 'etag': 'W/"4ecfeb39211c45f339a438a1268c0879"', 'href': 'https://lobste.rs/t/security.rss', 'status': 200, 'encoding': 'utf-8', 'version': 'rss20', 'namespaces': {'': 'http://www.w3.org/2005/Atom'}}

@iCiccio
Copy link

iCiccio commented Jun 22, 2022

I think that it is related also to #281, this is why it works for @n3s7or. I have the same issue that does not happen when I launch the script with an IDE or within the python interpreter, but it raises the error when I call the script from the terminal.

@iCiccio
Copy link

iCiccio commented Jun 23, 2022

I think that it is related also to #281, this is why it works for @n3s7or. I have the same issue that does not happen when I launch the script with an IDE or within the python interpreter, but it raises the error when I call the script from the terminal.

I think that I have identified the problem. It depends on the amount of active/opened connections by your script. You can handle this issue in two possible ways.
The first method (terminal):

$ ulimit -n 1048576
$ python your_script.py

The second method (in the script, I put it in the main file)

import resource
resource.setrlimit(resource.RLIMIT_OFILE, (1048576, 1048576))

where the number 1048576 is the integer that I choose.

I hope it will be useful

@vishalnandagopal
Copy link
Author

It depends on the amount of active/opened connections by your script. You can handle this issue in two possible ways.

But in the code I used (in the 1st comment) shouldn't the number of open connections be one since it is fetching only one URL?

@kurtmckee
Copy link
Owner

feedparser is migrating off custom HTTP client code and will rely exclusively on the requests package.

The new code has not been released yet, but because requests will be solely responsible for handling HTTP requests, I'm closing this as no longer relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants