[Solved-5 Solutions] urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error - Python

Error Description:



Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bokeh.sampledata
>>> bokeh.sampledata.download()
Using data directory: /Users/chad/.bokeh/data
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/chad/Library/Python/3.6/lib/python/site-packages/bokeh/sampledata/__init__.py", line 81, in download
    _getfile(base_url, file_name, data_dir, progress=progress)
  File "/Users/chad/Library/Python/3.6/lib/python/site-packages/bokeh/sampledata/__init__.py", line 87, in _getfile
    url = urlopen(file_url)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)> 

Solution 1:

Installing PyOpenSSL using pip will work without converting to PEM

 pip install PyOpenSSL

Solution 2:

  • If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, and can't validate any SSL connections.
  • This is a change for 3.6 on OSX, and requires a post-install step, which installs the certifi package of certificates.
  • This is documented in the ReadMe, which you should find at /Applications/Python\ 3.6/ReadMe.rtf
  • The ReadMe will have you run this post-install script, which just installs certifi: /Applications/Python\ 3.6/Install\ Certificates.command

Solution 3:

  • On Windows, Python does not look at the system certificate, it uses its own located at ?\lib\site-packages\certifi\cacert.pem.
  • The solution to our problem:
    • Download the domain validation certificate as *.crt or *pem file
    • Open the file in editor and copy it's content to clipboard
    • Find our cacert.pem location: from requests.utils import DEFAULT_CA_BUNDLE_PATH; print(DEFAULT_CA_BUNDLE_PATH)
    • Edit the cacert.pem file and paste our domain validation certificate at the end of the file.
    • Save the file and enjoy requests!

Solution 4:

  • If we just want to bypass verification, we can create a new SSLContext. By default newly created contexts use CERT_NONE
  • When calling the SSLContext constructor directly, CERT_NONE is the default.
  • Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time we would like to ensure the authenticity of the server we’re talking to.
  • Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED.
  • But if we want it to work for some other reason you can do the following, we'll have to import ssl as well:
input = input.replace("!web ", "")      
url = "https://domainsearch.p.mashape.com/index.php?name=" + input
req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' })
gcontext = ssl.SSLContext(ssl.PROTOCOL_TLSv1)  # Only for gangstars
info = urllib2.urlopen(req, context=gcontext).read()
Message.Chat.SendMessage ("" + info) 
  • This should get round our problem but we're not really solving any of the issues, but we won't see the [SSL: CERTIFICATE_VERIFY_FAILED] because we now aren't verifying the cert!
  • This PEP proposes to enable verification of X509 certificate signatures, as well as hostname verification for Python's HTTP clients by default, subject to opt-out on a per-call basis. This change would be applied to Python 2.7, Python 3.4, and Python 3.5.
import ssl
# This restores the same behavior as before.
context = ssl._create_unverified_context()
urllib.urlopen("https://no-valid-cert", context=context) 
  • It also features a highly discouraged option via monkeypatching which we don't often see in python:
import ssl

ssl._create_default_https_context = ssl._create_unverified_context 
  • Which overrides the default function for context creation with the function to create an unverified context.

Solution 5:

  • In Python 3.6.1 on MacOs Sierra

Entering this in the bash terminal solved the problem:

pip install certifi
/Applications/Python\ 3.6/Install\ Certificates.command 

Related Searches to urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error