Package Manager Conda Finds Space in Python’s Burrow
A year after the launch of ‘Conda’, Travis Oliphant, the founder of Anaconda, wrote in a blog about how he was often asked why he was promoting another Python package manager like Conda instead of just contributing to the standard Python packages. But that’s exactly what Oliphant had done. Conda came around after a meeting with Python founder Guido van Rossum in 2012 at the inaugural for PyData. Rossum was answering Oliphant’s questions around packaging issues and stated, “it really sounds like your needs are so unusual compared to the larger Python community that you’re better off building your own”. Travis Oliphant, creator of SciPy, NumPy, and Numba, founder and Director of Anaconda, Inc. Why use Conda at all? Conda was introduced as an alternative to Python’s native package manager pip that comes pre-installed with the most recent versions of Python. But Conda hasn’t gathered sufficient acclaim for itself to justify its inevitability. Author and MLOps expert Noah Gift questioned how popular Conda was among AWS Sagemaker users. What good would it do to install Conda when pip was already installed? The collective response to this is as scattered as it could be. A deep learning researcher and PhD scholar Sebastian Raschka stated that he preferred the Conda environment after having tried all other combinations. “Pip and virtualenv allow you to install packages in virtual environments with different requirements, but it doesn’t help you do the same with different Python version. For that, you’d need pyenv (another tool for managing different versions of Python) on top of that, and that can get pretty messy real quick if you are not careful,” he noted. Conda’s convenient advantages The convenience offered by Conda is one of its biggest strengths hailed by the community. Developers can install their packages and manage the environments all within the same tool while, with pip, users have to use another external tool on top of it to manage the environments. Another developer weighed in saying, “Based on my experience with Conda, it is the easiest way to install packages, especially if they depend on native code. Conda, in my view, is also the easiest way to reuse environments across projects on your local machine. And I don’t know any alternatives that would be as convenient as Conda.” Source: StackOverflow Conda also has what is known as Anaconda Navigator, a GUI or Graphical User Interface-programme to launch widely used applications like Jupyter, RStudio and VSCode. The navigator makes life much easier for developers. Another MLOps engineer echoed this: “I have found the Conda environment very useful if you have a GPU machine and are working with multiple versions of Cuda on those projects”. Conda also comes with a bunch of ML packages like NumPy, Pandas, Scikit-Learn and Matplotlib preinstalled. A developer mentioned this on a Reddit community discussion saying, “It gives you all the standard packages used in scientific computing in a convenient package without having to worry about installing them all individually with their dependencies. If you don’t plan on using typical scientific computing packages or any of the packaged software, the only downside is that you end up downloading software that you might not need,” he noted. The main difference between the two is that while Conda is language-agnostic and a cross-platform environment manager, pip is a general-purpose manager for everything Python. If all your work is with Python packages within isolated environments, Conda and pip+virtualenv are mostly replaceable. Not competitors, Different target audiences Turns out the question between these two isn’t which is better, it’s simply that the user base these two cater to is different—If a user wants to manage Python packages within an existing Python installation, Conda is useless because it will only be able to install packages within a Conda environment. If the user wants to work with Python packages that are normally have external dependencies like NumPy, pip won’t work because it is only restricted to Python packages. This does suggest that Conda and pip aren’t in the same competition as was earlier assumed. But pip does have the same advantages that any tool with a massive pre-existing community has—users who prefer the standard venv or virtualenv over Conda will be more likely to find solutions to their queries or errors they find on StackOverflow simply because pip is used by more people already. The other merit that pip has is that it has already been more battle-tested and is consequently believed to be more robust. Conda might have errors that users may have never found with pip, building the perception that pip is simply more reliable. A number of developers explicitly stated that their new go-to package is Miniforge which is essentially ‘Miniconda’, a much smaller sibling distribution for Anaconda or prefer using Miniconda itself. Miniforge has conda-forge as the default package. A developer commented saying Miniforge is “way better tested for OS and version compatibility compared to the average pip package”. Another developer lauded Miniforge stated, “While the Anaconda team maintains the core scientific stack you get when you install Anaconda, the ‘conda-forge’ channel includes a lot of the other major Python packages not included in Anaconda.”
Turns out the question between these two isn’t which is better, it’s simply that the user base these two cater to is different.
Turns out the question between these two isn’t which is better, it’s simply that the user base these two cater to is different.
[]
["scikit-learn", "NumPy", "AI", "ML", "MLOps", "Aim", "deep learning", "Matplotlib", "Jupyter", "Pandas"]
["AI", "ML", "deep learning", "MLOps", "Aim", "scikit-learn", "Jupyter", "Pandas", "NumPy", "Matplotlib"]
Poulomi is a Technology Journalist with Analytics India Magazine. Her fascination with tech and eagerness to dive into new areas led her to the dynamic world of AI and data analytics.