I am currently using kedro, version 0.15.4
with pipenv, version 2018.11.26
.
At the moment, I have to do the following if I want to use Pipenv
(For this example, I want this project to reside in the kedro-pipenv
directory):
mkdir kedro-pipenv && cd kedro-pipenv
- This is needed so that the
virtualenv
created is "tied" to the project directory which only really means that it has a name that is based on the directory name from which the pipenv install kedro
or pipenv shell
commands are executed.
- Unfortunately, at the moment
Pipenv
doesn't have the functionality to support custom virtualenv
names.
pipenv install kedro
- In my example, this generates a
virtualenv
with name kedro-pipenv-AB9IGRnB
which resides in the following location ~/.local/share/virtualenvs/kedro-pipenv-AB9IGRnB/
pipenv shell
kedro info
- all is looking good,
kedro
was successfully installed in the virualenv
handled by Pipenv
cd .. & kedro new
- I navigate one directory up and provide
kedro-pipenv
as the directory name for the project. Given that the directory was already created before in step 1, this fails which is expected and I get the following message:
cookiecutter.exceptions.OutputDirExistsException:
Error: "/Users/xyz/projects/kedro-pipenv" directory already exists
Run with --verbose to see the full exception
Error: Failed to generate project.
In order to "work around" this, I do the following whilst still in the same virtualenv
as before:
mv kedro-pipenv kedro-pipenv_tmp
- rename the existing directory
kedro new
- I provide
kedro-pipenv
as the directory name for the project.
mv kedro-pipenv_tmp/Pipfile* kedro-pipenv && rm -rf kedro-pipenv_tmp
- This step is needed so that the Python version and the single
kedro
dependency is maintained.
cd kedro-pipenv
kedro install
- Using
kedro build-reqs
for managing project requirements.
I am familiar with conda
as well and it seems that it is a much cleaner way of handling environments for Kedro at the moment.
However, for most of my other projects I have been using pyenv
in conjunction with Pipenv
for environment and dependency management. This allows me to have environment information tied to specific project spaces by having a Pipfile
in each of my projects' root directory.
Has anyone got any suggestions on how to improve the above workflow?