librelist archives

« back to archive

Infrastructure for a correctly designed package index

Infrastructure for a correctly designed package index

From:
tom Tom
Date:
2012-06-15 @ 00:07
In the README it says that " Infrastructure for a correctly designed
package index, using well-known packaging practices instead of the broken
easy_install + pypi model (easy mirroring, enforced metadata, indexing to
enable querying-before-installing, reliable install, etc...)." is a planned
feature. Has this feature been worked on or discussed before?

I think a more modern packaging system needs to be created, I think the
pypi+easy_install model is broken and the PyPi web interface horrible to
use. I think it wouldn't be too hard to kill two birds with one stone and
create much better alternatives.

Does anyone have any ideas or feelings about this?

Re: [bento] Infrastructure for a correctly designed package index

From:
David Cournapeau
Date:
2012-06-15 @ 08:44
Hi Tom,

On Fri, Jun 15, 2012 at 1:07 AM, tom Tom <tom@tomforb.es> wrote:

> In the README it says that " Infrastructure for a correctly designed
> package index, using well-known packaging practices instead of the broken
> easy_install + pypi model (easy mirroring, enforced metadata, indexing to
> enable querying-before-installing, reliable install, etc...)." is a planned
> feature. Has this feature been worked on or discussed before?
>

I played a bit with local install/uninstall feature, but nothing very
advanced. There has been some related discussion there:

https://groups.google.com/forum/?fromgroups#!topic/numfocus/MnRzBhmqXqk

For local install/uninstall, the main issue is how to support existing
installations and non-bento packages. Some of the things I am considering:
  - virtualenv-like program which would then have its own database of
installed/uninstalled packages. It would be nice to have a virtualenv
option that allows for bento instead of distribute or setuptools if
required by the user.
  - an installer should have the ability to handle binary installers. In
principle, going between e.g. msi/exe/egg on windows should be possible.
  - handling existing distutils programs could be done in some poor-man
sandbox
  - the most important: I need to find a 3-4 letters name for the installer
:)

For a remote service, I have thought even less about it
  - it should be based on a rest-like API that is documented
  - it should be easy to deploy locally for mirroring

I wonder if we could just use couchdb for the metadata (like the Node.js
package manager does).

What I know for sure is that I won't be able to do it all :) It would be
interesting to have people playing around ideas there.

David

Re: [bento] Infrastructure for a correctly designed package index

From:
tom Tom
Date:
2012-06-15 @ 10:57
The name should be the easy part: snake or boa for example?

I'm more interested in the remote service than the local install/uninstall.
Creating a easily redeployable service with a documented API would be easy
as pie with a framework like Django and a few extensions.
I might do some digging through the PyPi source code later to get a better
understanding of how it works internally.

The issue is really if these tools are needed. The current ones may be
conceptually broken but they still function and are incredibly popular, not
to mention being pretty ingrained in both peoples minds and toolchains.
Would creating alternatives be worth the effort?

~Tom

On Fri, Jun 15, 2012 at 9:44 AM, David Cournapeau <cournape@gmail.com>wrote:

> Hi Tom,
>
> On Fri, Jun 15, 2012 at 1:07 AM, tom Tom <tom@tomforb.es> wrote:
>
>> In the README it says that " Infrastructure for a correctly designed
>> package index, using well-known packaging practices instead of the broken
>> easy_install + pypi model (easy mirroring, enforced metadata, indexing to
>> enable querying-before-installing, reliable install, etc...)." is a planned
>> feature. Has this feature been worked on or discussed before?
>>
>
> I played a bit with local install/uninstall feature, but nothing very
> advanced. There has been some related discussion there:
>
> https://groups.google.com/forum/?fromgroups#!topic/numfocus/MnRzBhmqXqk
>
> For local install/uninstall, the main issue is how to support existing
> installations and non-bento packages. Some of the things I am considering:
>   - virtualenv-like program which would then have its own database of
> installed/uninstalled packages. It would be nice to have a virtualenv
> option that allows for bento instead of distribute or setuptools if
> required by the user.
>   - an installer should have the ability to handle binary installers. In
> principle, going between e.g. msi/exe/egg on windows should be possible.
>   - handling existing distutils programs could be done in some poor-man
> sandbox
>   - the most important: I need to find a 3-4 letters name for the
> installer :)
>
> For a remote service, I have thought even less about it
>   - it should be based on a rest-like API that is documented
>   - it should be easy to deploy locally for mirroring
>
> I wonder if we could just use couchdb for the metadata (like the Node.js
> package manager does).
>
> What I know for sure is that I won't be able to do it all :) It would be
> interesting to have people playing around ideas there.
>
> David
>

Re: [bento] Infrastructure for a correctly designed package index

From:
David Cournapeau
Date:
2012-06-15 @ 11:34
On Fri, Jun 15, 2012 at 11:57 AM, tom Tom <tom@tomforb.es> wrote:

> The name should be the easy part: snake or boa for example?


I would rather have something around the bento/japanese cooking domain :)


>
> I'm more interested in the remote service than the local
> install/uninstall. Creating a easily redeployable service with a documented
> API would be easy as pie with a framework like Django and a few extensions.
> I might do some digging through the PyPi source code later to get a better
> understanding of how it works internally.
>
> The issue is really if these tools are needed. The current ones may be
> conceptually broken but they still function and are incredibly popular, not
> to mention being pretty ingrained in both peoples minds and toolchains.
> Would creating alternatives be worth the effort?
>

That's certainly a valid concern. Some of the things I am quite
dissatisfied with:
  - lack of enforced metadata and its consequences
  - lack of automatically tested packages
  - simple (discoverable and documented) web API
  - easy to mirror and deploy locally

Intuitively, I'd also like a more decentralized infrastructure that what we
have now.

David