pip - How can I install GRPCIO on an Apple M1 Silicon laptop?

Question

Every time I try to install a pip package with GRPCIO as a dependency inside a .direnv project I get a build failure due to my architecture being arm64. How can I work around this while waiting for the GRPCIO crew to release an update?

  Using cached grpcio-1.34.0.tar.gz (21.0 MB)
    ERROR: Command errored out with exit status 1:
     command: /Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py'"'"'; __file__='"'"'/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'
'"'"', '"'"'
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-pip-egg-info-z41yqiw6
         cwd: /private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py", line 359, in <module>
        if mac_target and (pkg_resources.parse_version(mac_target) <
      File "/Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 113, in parse_version
        return packaging.version.Version(v)
      File "/Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 275, in __init__
        match = self._regex.search(version)
    TypeError: expected string or bytes-like object
    ASM Builds for BoringSSL currently not supported on: macosx-11-arm64
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Answer 1

I found a solution that works taking from https://github.com/grpc/grpc/issues/25082 with more detail and the removal of certain unnecessary steps. Note that this only works on python version 3.9 as of this post date.

First, install pyenv and pyenv-virtualenv.

brew install pyenv pyenv-virtualenv

Note that you might need to futz with the shell evaluation variable as simply adding eval "$(pyenv init -)" does not seem to work on the Apple M1 computers. Instead, run the following

pyenv init - >> .zshrc
pyenv virtualenv-init - >> .zshrc

Then go to your python project directory and create an .envrc and register it with direnv

echo 'layout pyenv 3.9.1' > .envrc
direnv allow

Next there install your requirements.txt via

GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install -r requirements.txt

Collecting google-cloud-pubsub
  Using cached google_cloud_pubsub-2.3.0-py2.py3-none-any.whl (210 kB)
Collecting proto-plus>=1.7.1
  Using cached proto_plus-1.17.0-py3-none-any.whl (42 kB)
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3
  Using cached grpc-google-iam-v1-0.12.3.tar.gz (13 kB)
Collecting libcst>=0.3.10
  Using cached libcst-0.3.17-py3-none-any.whl (507 kB)
Collecting google-api-core[grpc]<2.0.0dev,>=1.22.2
  Using cached google_api_core-1.26.1-py2.py3-none-any.whl (92 kB)
Collecting protobuf>=3.12.0
  Using cached protobuf-3.15.6-py2.py3-none-any.whl (173 kB)
Collecting grpcio<2.0.0dev,>=1.0.0
  Using cached grpcio-1.36.1.tar.gz (21.5 MB)
Collecting googleapis-common-protos[grpc]<2.0.0dev,>=1.5.2
  Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB)
Collecting typing-inspect>=0.4.0
  Using cached typing_inspect-0.6.0-py3-none-any.whl (8.1 kB)
Processing /Users/rkmacmini/Library/Caches/pip/wheels/b7/a5/c4/504d913c2a55bb09c607541578ec5f844d1ff33467abe93ba5/PyYAML-5.4.1-cp39-cp39-macosx_11_0_arm64.whl
Collecting typing-extensions>=3.7.4.2
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting six>=1.13.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting google-auth<2.0dev,>=1.21.1
  Using cached google_auth-1.27.1-py2.py3-none-any.whl (136 kB)
Collecting pytz
  Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Requirement already satisfied: setuptools>=40.3.0 in ./.direnv/python-3.9.1/lib/python3.9/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.22.2->google-cloud-pubsub->-r requirements.txt (line 1)) (49.2.1)
Collecting packaging>=14.3
  Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)
Collecting requests<3.0.0dev,>=2.18.0
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting mypy-extensions>=0.3.0
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.1-py3-none-any.whl (12 kB)
Collecting rsa<5,>=3.1.4; python_version >= "3.6"
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting idna<3,>=2.5
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.4-py2.py3-none-any.whl (153 kB)
     |████████████████████████████████| 153 kB 1.1 MB/s
Collecting certifi>=2017.4.17
  Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting chardet<5,>=3.0.2
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Using legacy 'setup.py install' for grpc-google-iam-v1, since package 'wheel' is not installed.
Using legacy 'setup.py install' for grpcio, since package 'wheel' is not installed.
Installing collected packages: six, protobuf, proto-plus, grpcio, googleapis-common-protos, grpc-google-iam-v1, mypy-extensions, typing-extensions, typing-inspect, pyyaml, libcst, pyasn1, pyasn1-modules, cachetools, rsa, google-auth, pytz, pyparsing, packaging, idna, urllib3, certifi, chardet, requests, google-api-core, google-cloud-pubsub
    Running setup.py install for grpcio ... done
    Running setup.py install for grpc-google-iam-v1 ... done
Successfully installed cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 google-api-core-1.26.1 google-auth-1.27.1 google-cloud-pubsub-2.3.0 googleapis-common-protos-1.53.0 grpc-google-iam-v1-0.12.3 grpcio-1.36.1 idna-2.10 libcst-0.3.17 mypy-extensions-0.4.3 packaging-20.9 proto-plus-1.17.0 protobuf-3.15.6 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-2.4.7 pytz-2021.1 pyyaml-5.4.1 requests-2.25.1 rsa-4.7.2 six-1.15.0 typing-extensions-3.7.4.3 typing-inspect-0.6.0 urllib3-1.26.4

NOTE: Some references say you need to point your CFLAGS, etc, at the brew installed openssl locations but that has not seemed necessary for me. If you have issue you can try the command with the CFLAGS enabled via

CFLAGS="-I /opt/homebrew/opt/openssl/include" LDFLAGS="-L /opt/homebrew/opt/openssl/lib" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install -r requirements.txt