[05:03:53] <dude-x> huh. installed pip 20.1 thought i'd try the new beta resolver to upgrade a few packages. nothing gets upgraded. :/
[18:26:52] <Raito_Bezarius> Hello, anyone knows why https://files.pythonhosted.org/packages/py2.py3/r/requests-oauthlib/requests_oauthlib-1.3.0-py2.py3-none-any.whl is a 404 though on PyPI, it seems like such a file should exist (?)
[18:31:25] <tos9> Raito_Bezarius: You almost certainly aren't really supposed to hotlink to random files on PyPI in that way -- is that what you're trying to do there?
[18:32:21] <ngoldbaum> that file is available at https://files.pythonhosted.org/packages/a3/12/b92740d845ab62ea4edf04d2f4164d82532b5a0b03836d4d4e71c6f3d379/requests_oauthlib-1.3.0-py2.py3-none-any.whl FWIW
[18:32:27] <tos9> Raito_Bezarius: That wheel does appear here: https://pypi.org/project/requests-oauthlib/#files, so if you're .. what ngoldbaum said :)
[18:32:41] <tos9> Basically pip will find it. Whether it has some hotlink-available version I doubt is guaranteed
[18:32:44] <ngoldbaum> i don't think there's a stable URL scheme for wheels
[18:32:48] <tos9> But if it is,... also what ngoldbaum said :D
[18:32:51] <Raito_Bezarius> tos9: I don't have the choice
[18:32:52] <ngoldbaum> there is one for the sdist though
[18:37:15] <ngoldbaum> it may be the case that URL scheme worked at one point
[18:37:19] <ngoldbaum> but i don't think it works now...
[18:37:19] <Raito_Bezarius> Is it a "You cannot" due to arbitrary limitations in PyPI/pip or "You cannot" due to "it's a bit complex"?
[18:37:34] <tos9> Raito_Bezarius: Nothing is ever arbitrary about limitations :)
[18:37:35] <ngoldbaum> e.g. pypi moved to the URLs with the hashes for improved security a while ago
[18:37:36] <Raito_Bezarius> ngoldbaum: Is there any reason for the URL scheme to require those xx/yy/zzz, are those computable?
[18:37:48] <ngoldbaum> not without having the file already i think
[18:37:50] <Raito_Bezarius> I'm okay with hashes in the URL, but are those hashes documented?
[18:38:41] <ngoldbaum> there's a pep where all this is documented, i don't have a reference for it off the top of my head
[18:39:01] <Raito_Bezarius> https://www.python.org/dev/peps/pep-0503/ this one?
[18:39:47] <Raito_Bezarius> ah maybe this one https://www.python.org/dev/peps/pep-0458/
[18:40:25] <ngoldbaum> i'm not sure, also keep in mind that there are peps that are in the database but where never implemented or have since been superceded
[18:40:39] <ngoldbaum> unfortunately keeping all this straight approaches full-time job :/
[18:41:35] <Raito_Bezarius> I'm just trying to see how feasible it is to get metadata on PyPI, because that would make Python packaging on Nix so simple
[18:41:57] <Raito_Bezarius> ngoldbaum: there must be some way for pip to find those URLs, right?
[18:42:29] <ngoldbaum> i don't know but also keep in mind that as tos9 told you already they don't want you to reimplement what pip does
[18:42:44] <ngoldbaum> it's probably a lot easier just to use pip for whatever you're trying to do
[18:44:00] <Raito_Bezarius> i'd love to, but pip is just not doing the job alas
[18:44:12] <Raito_Bezarius> maybe in the future, with the new dependency resolving & stuff
[18:44:22] <ngoldbaum> you can tell pip to not install dependencies of a package
[18:44:27] <tos9> Raito_Bezarius: You're making incorrect assumptions
[18:45:37] <ngoldbaum> where does a lockfile come in?
[18:45:55] <Raito_Bezarius> Sorry, the discussion just got confusing, I'll summary what I'm trying to do
[18:46:04] <tos9> Raito_Bezarius: That's exactly what I was hoping you'd do!
[18:47:18] <Raito_Bezarius> Given a requirements.txt or pyproject.toml, I'd like to ensure that dependencies are pinned (e.g. I need a lockfile, right?) and I want to be able to install them as a derivation from Nix, I cannot just do `pip install XXX` because it'll use Internet to install dependencies and I don't want to use fixed-output derivation because they're cumbersome for development
[18:47:33] <Raito_Bezarius> Also, I would like to install wheels and not sdist
[18:47:36] <Raito_Bezarius> Because of tensorflow & friends
[18:48:20] <Raito_Bezarius> (let us ignore special cases such as local URLs in requirements.txt, editable and git URLs for example.)
[18:49:26] <Raito_Bezarius> So what I tried is to use poetry2nix which uses the lockfile from Poetry to download the dependencies (and bypass pip I guess), the thing is that as you said, I cannot rely on the URL scheme for security reasons
[18:49:34] <ngoldbaum> isn't a requirements.txt with pinned dependencies exactly what you want?
[18:49:43] <Raito_Bezarius> I'm pretty sure the data in the lockfile from Poetry includes enough information to rebuild the hashes that PyPI uses
[18:49:58] <Raito_Bezarius> ngoldbaum: Well, that was until I encountered dependencies which installed contradictory versions
[18:50:16] <Raito_Bezarius> I didn't investigate really, but I'm okay with requirements.txt if it's enough
[18:50:20] <ngoldbaum> then the requirements.txt didn't have all the packages in the virtualenv
[18:51:10] <Raito_Bezarius> That's possible, I preferred to be on the safe side with the poetry lock
[18:51:12] <ngoldbaum> you want to have a listing of all of your full python dependency tree along with the versions of the packages, pip freeze in a virtualenv will give you that
[18:51:22] <Raito_Bezarius> but pip freeze require to have it already installed
[18:56:00] <Raito_Bezarius> I want to do the download phase, hash check phase, caching phase, myself
[18:56:22] <ngoldbaum> but of wheels, which is a little weird
[18:56:23] <Raito_Bezarius> And for the installation phase, I'd prefer to delegate to pip or whatever which has implemented more of PEPXXX than me regarding "installation mechanisms in Python"
[20:45:57] <Raito_Bezarius> The only question I have left, which I asked at #pypa-dev, is: why using blake2_256 when sha256 is used everywhere else (and only available to Simple API)
[20:46:20] <Raito_Bezarius> "curl https://pypi.org/pypi/requests-oauthlib/json" don't expose any blake2_256 hash
[21:18:56] <energizer> does pypi have any data about how often packages are installed using Poetry?
[21:31:21] <dstufft> energizer: does it have a unique user agent?
[21:38:21] <energizer> dstufft: i think no, it only sets the user agent when uploading. for downloading i think it wraps pip. does pip have a way to set the user agent?
[21:39:19] <energizer> oh i see, yes. PIP_USER_AGENT_USER_DATA
[21:52:38] <dstufft> such a silly feature lol, but w/e - that doesnt really help if poetry is using pip until they populate that field and pypi is updated to understand it
[21:53:48] <dstufft> in anycase, i dont remembe what uas we support atm, but given a unique ua we can track metric against that