最新消息:20181230 VPS服务器已从Linode换到腾讯云香港,主题仍用朋友推荐的大前端D8

【已解决】pipenv install PySpider卡死在:Locking [packages] dependencies

pyspider crifan 444浏览 0评论

折腾:

【已解决】pipenv虚拟环境中用pip安装pyspider出错:__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using

期间,虽然貌似可以继续安装了,但是结果卡死在:

<code>➜  xxx pipenv install pyspider
Installing pyspider…
Looking in indexes: https://pypi.python.org/simple
Collecting pyspider
Collecting chardet&gt;=2.2 (from pyspider)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting u-msgpack-python&gt;=1.6 (from pyspider)
  ...
  Running setup.py bdist_wheel for jsmin: started
  Running setup.py bdist_wheel for jsmin: finished with status 'done'
  Stored in directory: /Users/crifan/Library/Caches/pip/wheels/64/f4/de/9667d84f759289edf5442220997c6d4334637a6bb2a7b90f73
  Running setup.py bdist_wheel for PyYAML: started
  Running setup.py bdist_wheel for PyYAML: finished with status 'done'
  Stored in directory: /Users/crifan/Library/Caches/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f
Successfully built jsmin PyYAML
Installing collected packages: chardet, u-msgpack-python, click, Werkzeug, itsdangerous, MarkupSafe, Jinja2, Flask, Flask-Login, defusedxml, jsmin, PyYAML, wsgidav, tblib, six, lxml, tornado, cssselect, pyquery, certifi, idna, urllib3, requests, pyspider
Successfully installed Flask-1.0.2 Flask-Login-0.4.1 Jinja2-2.10 MarkupSafe-1.0 PyYAML-3.13 Werkzeug-0.14.1 certifi-2018.4.16 chardet-3.0.4 click-6.7 cssselect-1.0.3 defusedxml-0.5.0 idna-2.7 itsdangerous-0.24 jsmin-2.2.2 lxml-4.2.3 pyquery-1.4.0 pyspider-0.3.10 requests-2.19.1 six-1.11.0 tblib-1.3.2 tornado-4.5.3 u-msgpack-python-2.5.0 urllib3-1.23 wsgidav-2.4.1

Adding pyspider to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
</code>

pipenv stop at Locking [packages] dependencies

`pipenv install XYZ` command sometimes hangs indefinitely at “Locking [packages] dependencies” · Issue #1816 · pypa/pipenv

<code>➜  xx ll ~/.cache/.pip
ls: /Users/crifan/.cache/.pip: No such file or directory
➜  x ll ~/.cache/
total 0
drwxr-xr-x  4 crifan  staff   128B  8  4  2016 bower
➜  x ll ~/.cache/bower/packages/a5e5328356c7c0a256b8b56d0e278258/
➜  x ll ~/Library/Caches/pip
total 8
drwx------  18 crifan  staff   576B 12 26  2017 http
-rw-r--r--   1 crifan  staff   397B  7  5 13:59 selfcheck.json
drwxr-xr-x  43 crifan  staff   1.3K  7 11 10:19 wheels
➜  x rm -rf ~/Library/Caches/pip/
</code>

然后继续试试

问题依旧。

去把pipenv的cache也删了:

<code>➜  x rm ~/Library/Caches/pip
rm: /Users/crifan/Library/Caches/pip: is a directory
➜  x rm ~/Library/Caches/pipenv
rm: /Users/crifan/Library/Caches/pipenv: is a directory
➜  x ll ~/Library/Caches/pip
total 0
drwx------  18 crifan  staff   576B  7 11 10:33 http
➜  x ll ~/Library/Caches/pipenv
total 8
-rw-r--r--   1 crifan  staff   2.3K  7 11 10:22 depcache-py3.6.json
drwx------  18 crifan  staff   576B  4 18 14:19 hash-cache
drwxr-xr-x  13 crifan  staff   416B  7 11 10:22 pkgs
drwxr-xr-x  48 crifan  staff   1.5K  7 11 10:22 wheels
➜  x rm -rf ~/Library/Caches/pip
➜  x rm -rf ~/Library/Caches/pipenv
</code>

结果:

问题依旧。

然后:

<code>pipenv-resolver --debug pyspider
</code>

输出很多都显示在cache中,然后部分不在cache,会download下载

另外看到:

It just hangs · Issue #278 · pypa/pipenv

`pipenv install XYZ` command sometimes hangs indefinitely at “Locking [packages] dependencies” · Issue #1816 · pypa/pipenv

貌似最新版本的pipenv已经更新版本了?

pipenv install is very slow · Issue #356 · pypa/pipenv

实在不行,就:

重新删除这个pipenv

<code>➜  x pipenv --rm
Removing virtualenv (/Users/crifan/.local/share/virtualenvs/x-Ee60YgCa)…
</code>

然后重新试试

不过试试之前,去升级pipenv:

【已解决】Mac中升级Python 3的pipenv的版本

然后重新用最新的2018.7.1的pipenv去试试

好像直接卡死了。

<code>➜  x pipenv install pyspider
Creating a virtualenv for this project...
Pipfile: /Users/crifan/dev/dev_root/x/Pipfile
Using /usr/local/bin/python3.6m (3.6.4) to create virtualenv...
⠋^Environment creation aborted.
➜  x pipenv --rm
No virtualenv has been created for this project yet!
</code>

换个目录,重新再去试试。

有进展了:

结果也一直停在这里,好像卡死了

<code>➜  x git:(master) ✗ rm -rf ~/Library/Caches/pip/
➜  x git:(master) ✗ rm -rf ~/Library/Caches/pipenv
</code>

继续试试

问题依旧。

试试:

<code>pipenv-resolver --debug pyspider
</code>

停在了:

好像是网络问题?

那去全部翻墙ss,然后再去试试

<code>pipenv-resolver --debug pyspider
</code>

还是卡死在:

https://files.pythonhosted.org/packages/54/a6/43be8cf1cc23e3fa208cab04ba2f9c3b7af0233aab32af6b5089122b44cd/lxml-4.2.3.tar.gz#sha256=622f7e40faef13d232fb52003661f2764ce6cdef3edb0a59af7c1559e4cc36d1

去网页中是可以下载的:

<code>9d80bddc45c1f91cfe/lxml-4.2.3-cp36-cp36m-win_amd64.whl#sha256=2dc6705486b8abee1af9e2a3761e30a3cb19e8276f20ca7e137ee6611b93707c (from https://pypi.org/simple/lxml/), version: 4.2.3
DEBUG:notpip._internal.index:Found link https://files.pythonhosted.org/packages/d0/54/e62ee9e1d5131b42251257e087d3ff7769bd42ad9ee547cc02596407d79a/lxml-4.2.3-cp37-cp37m-manylinux1_i686.whl#sha256=6ec908b4c8a4faa7fe1a0080768e2ce733f268b287dfefb723273fb34141475f (from https://pypi.org/simple/lxml/), version: 4.2.3
DEBUG:notpip._internal.index:Found link https://files.pythonhosted.org/packages/de/4f/965d0a2469fe89c42424d22672075c0ce27d120470778c4d2fabee198e79/lxml-4.2.3-cp37-cp37m-manylinux1_x86_64.whl#sha256=9f0daddeefb0791a600e6195441910bdf01eac470be596b9467e6122b51239a6 (from https://pypi.org/simple/lxml/), version: 4.2.3
DEBUG:notpip._internal.index:Found link https://files.pythonhosted.org/packages/54/a6/43be8cf1cc23e3fa208cab04ba2f9c3b7af0233aab32af6b5089122b44cd/lxml-4.2.3.tar.gz#sha256=622f7e40faef13d232fb52003661f2764ce6cdef3edb0a59af7c1559e4cc36d1 (from https://pypi.org/simple/lxml/), version: 4.2.3
DEBUG:notpip._internal.index:Using version 4.2.3 (newest of versions: 0.9, 0.9.1, 0.9.2, 1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.3, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 2.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.1, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.2, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.2.6, 2.2.7, 2.2.8, 2.3, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 3.0, 3.0.1, 3.0.2, 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4, 3.2.5, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.3.5, 3.3.6, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4, 3.5.0, 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0,3.7.1, 3.7.2, 3.7.3, 3.8.0, 4.0.0, 4.1.0, 4.1.1, 4.2.0, 4.2.1, 4.2.2, 4.2.3)
</code>

DEBUG:notpip._internal.index:Found link lxml-4.2.3.tar.gz https://pypi.org/simple/lxml/version: 4.2.3

pipenv-resolver –debug pyspider DEBUG:notpip._internal.index:Found link lxml

pip – Python 包和依赖关系管理 – Python开发 – 评论 | CTOLib码库

Installing packages from internal PyPI indexes · Issue #1084 · pypa/pipenv

强行中断时看到貌似和ssl有关:

<code>DEBUG:notpip._internal.index:Found link https://files.pythonhosted.org/packages/54/a6/43be8cf1cc23e3fa208cab04ba2f9c3b7af0233aab32af6b5089122b44cd/lxml-4.2.3.tar.gz#sha256=622f7e40faef13d232fb52003661f2764ce6cdef3edb0a59af7c1559e4cc36d1 (from https://pypi.org/simple/lxml/), version: 4.2.3
^CTraceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 301, in get_legacy_dependencies
    wheel_cache=self.wheel_cache,
TypeError: __init__() got an unexpected keyword argument 'download_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/bin/pipenv-resolver", line 11, in &lt;module&gt;
    sys.exit(main())
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/resolver.py", line 76, in main
    system=system,
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/resolver.py", line 63, in resolve
    allow_global=system,
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/utils.py", line 402, in resolve_deps
    req_dir=req_dir
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/utils.py", line 293, in actually_resolve_deps
    results = resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/resolver.py", line 104, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/resolver.py", line 209, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/resolver.py", line 303, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 243, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 335, in get_legacy_dependencies
    self.resolver.resolve(reqset)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 107, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 264, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 214, in _get_abstract_dist_for
    self.require_hashes
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 314, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 838, in unpack_url
    progress_bar=progress_bar
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 675, in unpack_http_url
    progress_bar)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 899, in _download_http_url
    _download_url(resp, link, content_file, hashes, progress_bar)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 619, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/utils/hashes.py", line 48, incheck_against_chunks
    for chunk in chunks:
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 587, in written_chunks
    for chunk in chunks:
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 576, in resp_read
    decode_content=False):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 436,in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 384,in read
    data = self._fp.read(amt)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/cachecontrol/filewrapper.py", line 60, in read
    data = self.__fp.read(amt)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 449, in read
    n = self.readinto(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 493, in readinto
    n = self.fp.readinto(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1009, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 871, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
KeyboardInterrupt
</code>

估计是https的缘故了。

换个终端,问题依旧:

换成之前的:

<code>创建虚拟环境,再去用pip安装:
pip install pyspider
</code>

结果:

中间先去解决:

【已解决】pipenv虚拟环境中用pip安装pyspider出错:__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using

和之前类似,是可以正常下载和安装的。

(我猜测到最后可能还是会lock时卡死的)

<code>➜  xx git:(master) ✗ pip install pyspider
Collecting pyspider
  ...
  Using cached https://files.pythonhosted.org/packages/00/fd/5e65f293e366a63198dade275b886e5d24752367c2e67e3993023b0d58ef/lxml-4.2.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Requirement already satisfied: pycurl in /Users/crifan/.local/share/virtualenvs/xx-SW6GVzwk/lib/python3.6/site-packages (from pyspider) (7.43.0.2)
Collecting requests&gt;=2.2 (from pyspider)
  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)
    100% |████████████████████████████████| 92kB 64kB/s
...
  Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
    100% |████████████████████████████████| 276kB 108kB/s
Building wheels for collected packages: pyspider, Flask-Login, tornado, itsdangerous, MarkupSafe, jsmin, PyYAML
  Running setup.py bdist_wheel for pyspider ... done
  Stored in directory: /Users/crifan/Library/Caches/pip/wheels/39/60/ec/9ba1af9e0798333d32198784880b8cc5b22f00a81801c6fcec
 ...
  Running setup.py bdist_wheel for PyYAML ... done
  Stored in directory: /Users/crifan/Library/Caches/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f
Successfully built pyspider Flask-Login tornado itsdangerous MarkupSafe jsmin PyYAML
Installing collected packages: Werkzeug, MarkupSafe, Jinja2, click, itsdangerous, Flask, chardet, cssselect, lxml, certifi, urllib3, idna, requests, Flask-Login, u-msgpack-python, six, tblib, defusedxml, jsmin, PyYAML, wsgidav, tornado, pyquery, pyspider
Successfully installed Flask-1.0.2 Flask-Login-0.4.1 Jinja2-2.10 MarkupSafe-1.0 PyYAML-3.13 Werkzeug-0.14.1 certifi-2018.4.16 chardet-3.0.4 click-6.7 cssselect-1.0.3 defusedxml-0.5.0 idna-2.7 itsdangerous-0.24 jsmin-2.2.2 lxml-4.2.3 pyquery-1.4.0 pyspider-0.3.10 requests-2.19.1 six-1.11.0 tblib-1.3.2 tornado-4.5.3 u-msgpack-python-2.5.0 urllib3-1.23 wsgidav-2.4.1
</code>

这次竟然没有卡死。

去看Pipfile:

果然是空的

-》pip安装的话,没有同步到pipenv中

所以再去pipenv安装试试:

<code>pipenv install pyspider
</code>

-》确定了,就是lock出问题,install其实是可以正常安装的。

pipenv lock stopped working if –dev contains -e . on 11.6.2 · Issue #1708 · pypa/pipenv

<code>➜  xx git:(master) ✗ pipenv lock
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
</code>

Package locking is crazy slow for scikit-learn · Issue #1785 · pypa/pipenv

Cannot build lock even though dependencies are found. · Issue #931 · pypa/pipenv

Stop everything! Start using Pipenv! – bryson3Gps

【总结】

算了,既然暂时可以正常安装pyspide,那就先继续用吧。

或许过段时间pipenv的lock卡死的问题,官网自动就解决了呢。

【后记 20180723】

又试了试:

<code>Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 243, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 335, in get_legacy_dependencies
    self.resolver.resolve(reqset)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 107, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 264, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/resolve.py", line 214, in _get_abstract_dist_for
    self.require_hashes
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/operations/prepare.py", line 314, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 838, in unpack_url
    progress_bar=progress_bar
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 675, in unpack_http_url
    progress_bar)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 899, in _download_http_url
    _download_url(resp, link, content_file, hashes, progress_bar)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 619, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/utils/hashes.py", line 48, in check_against_chunks
    for chunk in chunks:
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 587, in written_chunks
    for chunk in chunks:
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_internal/download.py", line 576, in resp_read
    decode_content=False):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 307, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
</code>

`pipenv install XYZ` command sometimes hangs indefinitely at “Locking [packages] dependencies” · Issue #1816 · pypa/pipenv

去试试

<code>➜  x git:(master) ✗ pipenv --clear
Error: no such option: --clear
</code>

然后:

<code>➜  x git:(master) ✗ pipenv lock --clear
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

</code>

问题依旧。

<code>➜  x git:(master) ✗ pipenv lock -v
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
using sources: [{'url': 'https://pypi.org/simple', 'verify_ssl': True, 'name': 'pypi'}]
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  pyspider (from -r /var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pipenv-2e_ne979-requirements/pipenv-cktjawyn-constraints.txt (line 2))

Finding the best candidates:
  found candidate pyspider==0.3.10 (constraint was &lt;any&gt;)

Finding secondary dependencies:
  pyspider==0.3.10 not in cache, need to check index
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  pyspider (from -r /var/folders/46/2hjxz38n22n3ypp_5f6_p__00000gn/T/pipenv-2e_ne979-requirements/pipenv-04eb3dgu-constraints.txt (line 2))

Finding the best candidates:
  found candidate pyspider==0.3.10 (constraint was &lt;any&gt;)

Finding secondary dependencies:
  pyspider==0.3.10 not in cache, need to check index

CRITICAL:notpip._internal.index:Could not find a version that satisfies the requirement requests&gt;=2.2 (from pyspider==0.3.10) (from versions: )
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
No matching distribution found for requests&gt;=2.2 (from pyspider==0.3.10)
Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 301, in get_legacy_dependencies
    wheel_cache=self.wheel_cache,
TypeError: __init__() got an unexpected keyword argument 'download_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 302, in _error_catcher
    yield
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/cachecontrol/filewrapper.py", line 60, in read
    data = self.__fp.read(amt)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 449, in read
    n = self.readinto(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 493, in readinto
    n = self.fp.readinto(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1009, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 871, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/resolver.py", line 87, in &lt;module&gt;
    main()
...
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/crifan/Library/Python/3.6/lib/python/site-packages/pipenv/patched/notpip/_vendor/urllib3/response.py", line 307, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
</code>

感觉是ssl连接超时?导致了此处lock失败?

此处pipfile:

<code>➜  x git:(master) ✗ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyspider = "*"

[dev-packages]

[requires]
python_version = "3.6"
</code>

pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host=’files.pythonhosted.org‘, port=443): Read timed out.

pipenv pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host=’files.pythonhosted.org‘, port=443): Read timed out.

pip install Error – ReadTimeoutError: HTTPSConnectionPool(host=’pypi.python.org’, port=443): Read – CSDN博客

Cannot install pip packages · Issue #3826 · pypa/warehouse

Skipping page https://pypi.org/simple/idna/ because of Content-Type: unknown · Issue #5345 · pypa/pip

pipenv doesn’t respect pip.conf · Issue #856 · pypa/pipenv

pipenv 怎么写 lock 文件怎么这么慢?每次 install 都要卡在 Locking [packages] dependencies…这里好久 – V2EX

pipenv pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError host=files.pythonhosted.org

pipenv pipenv.patched.notpip._vendor.urllib3.exceptions.ReadTimeoutError

pipenv lock …. Timing out · Issue #1274 · pypa/pipenv

Exception raised creating Pipfile.lock · Issue #733 · pypa/pipenv

<code>➜  xx git:(master) ✗ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyspider = "*"

[dev-packages]

[requires]
python_version = "3.6"
</code>

把url改为:

https://pypi.tuna.tsinghua.edu.cn/simple

<code>[[source]]
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
verify_ssl = true
name = "pypi"

[packages]
pyspider = "*"

[dev-packages]

[requires]
python_version = "3.6"

</code>

结果:

<code>➜  xx git:(master) ✗ pipenv install pyspider
Installing pyspider...
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: pyspider in 
...
Requirement already satisfied: certifi&gt;=2017.4.17 in /Users/crifan/.local/share/virtualenvs/x-8_TdeiGn/lib/python3.6/site-packages (from requests&gt;=2.2-&gt;pyspider) (2018.4.16)

Adding pyspider to Pipfile's [packages]...
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (24eee4)!
Installing dependencies from Pipfile.lock (24eee4)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 25/25 — 00:00:10
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
</code>

就可以了。

pipenv HTTPSConnectionPool(host=’files.pythonhosted.org‘, port=443): Read timed out.

[Python]Python pip用国内镜像下载的办法 – CSDN博客

有机会再去试试:

<code>pip  install --index https://pypi.mirrors.ustc.edu.cn/simple/ pandas
</code>

Install Scrapy timeout using Pipenv · Issue #2392 · pypa/pipenv

说是修改PIPENV_TIMEOUT增加超时时间

[Help] after pypi change pipenv is unsuable when behind proxy. · Issue #2011 · pypa/pipenv

Error: HTTPSConnectionPool(host=’pypi.python.org’, port=443): Read timed out. – when running pip.exe -r requirements.txt | Odoo

【总结】

此处通过换source,把Pipfile中:

<code>url = "https://pypi.org/simple"
</code>

换成:

<code>url = "https://pypi.tuna.tsinghua.edu.cn/simple"
</code>

然后:

<code>pipenv install pyspider
</code>

就可以正常安装,且lock,不会死掉了。

至于根本原因到底是pipenv的问题,还是ssl的问题,还是之前的pypi.org的问题,暂时懒得深究了。

希望过段时间,pipenv最新版本,能自动解决此问题。

【后记2】

确定就是pipenv的lock的问题。

之前尝试了N多办法,也无法解决。

最终是升级到最新版,截止20190125,大多数情况下,都是正常的。不过偶尔还会出现lock耗时几十分钟的事情。

转载请注明:在路上 » 【已解决】pipenv install PySpider卡死在:Locking [packages] dependencies

发表我的评论
取消评论

表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
63 queries in 0.106 seconds, using 9.50MB memory