PMXBOT Log file Viewer

server donated by Rackspace

Help | Karma | Search:

#pypa logs for Tuesday the 16th of June, 2020

(Back to #pypa overview) (Back to channel listing) (Animate logs)
[01:18:20] <elibrokeit> FFY00: (also note that .get_all() is an undocumented implementation detail, and you're only ever supposed to use dictionary methods/syntax. So I'm genuinely unsure what the expected API for this is, especially for things which unlike requires-dist, don't have special properties...)
[07:59:43] <pradyunsg> There's no installation database entry for which extras are installed.
[11:22:45] <elibrokeit> pradyunsg: the idea would be to add something like the check_dependencies() function to some thematically appropriate yet central module. Nothing to do with installation database entries for extras.
[11:23:07] <elibrokeit> On the rationale that as seen here, it's harder than it looks to do this right...
[11:23:21] <elibrokeit> and it's annoyingly wordy to boot
[11:30:39] <pradyunsg> Basically, something like `pip check`?
[11:31:01] <pradyunsg> elibrokeit: ^
[13:21:14] <PSFSlack2> <deveshkusingh> pradyunsg: \o
[13:22:28] <PSFSlack2> <deveshkusingh> I had few queries from you about the triager section of https://pip.pypa.io/en/stable/development/contributing/#becoming-a-maintainer
[13:22:47] <PSFSlack2> <deveshkusingh> Let me know if you want me to PM you on IRC with those questions
[13:41:47] <elibrokeit> pradyunsg: something like that, yeah
[13:42:18] <pradyunsg> @deveshkusingh go ahead.
[13:44:41] <elibrokeit> would be nice to have some sort of pythonic API for this, that accepted requirement strings
[13:46:31] <pradyunsg> elibrokeit: FWIW, I (and many others here) want to break out all of the "logic" for nearly everything in pip into reusable libraries.
[13:47:45] <elibrokeit> pradyunsg: I know :)
[13:48:01] <elibrokeit> wondering where a good place to store this would be, though.
[13:48:14] <pradyunsg> Ask the importlib.metadata folks?
[13:49:55] <pradyunsg> I'm personally working on installer for a bit now. IMO the other three main areas that need work/better understanding/docs are: (1) "build" logic (2) environment introspection + installation database (think lockfiles, descriptions of environments and the overlap of the two) and (3) index/VCS interaction.
[13:50:53] <pradyunsg> importlib.metadata is basically what we need for (2) as far as I can tell. I don't think pip or anyone else is using them *yet*, but we all almost certainly want to.
[13:53:08] <pradyunsg> (1) is... well, I'd started with that and have stepped away from that, since it's been really difficult to make progress because of the sheer number of "do not do X" compared to "I want Y" responses in that discussion. :)
[13:54:00] <elibrokeit> lol, hmm
[13:54:35] <pradyunsg> I'd be glad if FFY00 is able to help us move forward on that front. I have seen some of the folks have changed their positions on things in that discussion, so it should be easier to see that through now AFAIK. :)
[13:54:49] <elibrokeit> importlib.{metadata,resource} would hopefully one day do everything pkg_resources does today, except without the steep, steep cost of an upfront workingset
[13:55:40] <pradyunsg> (3) is probably the easiest / most niche but, tbh. pip's internals have been refactored pretty thoroughly, and it shouldn't be too difficult to see that through.
[13:56:18] <pradyunsg> elibrokeit: they do almost everything already, I think.
[13:56:47] <elibrokeit> well, a lot of it, yes
[13:57:05] <pradyunsg> And anything that's missing, I'm sure @jaraco and other maintainers would be happy to accommodate reasonable requests.
[13:57:42] <pradyunsg> They'll decide what is "reasonable", of course :P
[13:57:58] <elibrokeit> pkg_resources.load_entry_point has its thing where it automatically aborts with DistributionNotFound if any requirements are missing, so *somewhere* in there it already knows how to check dependencies.
[13:58:12] <pradyunsg> lol
[13:58:22] <pradyunsg> Maybe. 🤷🏻‍♂️
[13:59:20] <jaraco> What is the question?
[13:59:41] <pradyunsg> adding something like pip check as an API to importlib.metadata
[13:59:47] <pradyunsg> jaraco: ^
[14:02:16] <elibrokeit> it would be useful to be able to verify if a requirements string is satisfied in the current environment, keeping in mind that foo[bar] should verify that foo's extra "bar" is actually available
[14:03:46] <jaraco> I think I've discussed this before and the underlying issue is that importlib.metadata is not at the right scope to do that work. It's a lower-level library and doesn't have access to things like `packaging` to parse requirements (much less evaluate their validity).
[14:04:13] <jaraco> Right - https://github.com/pypa/packaging-problems/issues/317
[14:04:59] <FFY00> jaraco, that's why I suggested this for pypa/packaging
[14:05:07] <elibrokeit> the current solution seems to be "use importlib_metadata to find foo, get its version, and pass that to packaging.requirements.Requirement(...).specifier.contains()", and pray that extras don't matter
[14:05:17] <FFY00> checking the dependencies is not really the job of importlib
[14:05:31] <elibrokeit> hence my thought "I'm not sure where this belongs"
[14:05:33] <FFY00> IMO it should be in packaging
[14:05:53] <jaraco> Yeah, the pypa pretty much settled on packaging. Now we need an implementation.
[14:06:00] <jaraco> > pray that extras don't matter
[14:06:02] <jaraco> I wouldn't say that.
[14:06:37] <jaraco> Extras should be supported. If importlib.metadata needs more sophistication to support that, I'd be eager to shepherd that support.
[14:06:39] <FFY00> I have a working implementation that should work with the extras
[14:06:51] <FFY00> in fact, it should verify the only dependency tree
[14:07:05] <FFY00> *verify the whole dependency tree
[14:07:40] <FFY00> verifying only part of the dependency tree is more code
[14:07:45] <FFY00> :P
[14:08:07] <jaraco> Some use-cases may wish to only verify a portion of the dependency tree (the portion they care about) and ignore other broken dependencies.
[14:08:34] <jaraco> But something is better than nothing.
[14:08:52] <jaraco> Unfortunately, pkg_resources isn't nothing, so there's a working implementation to supersede.
[14:11:20] <elibrokeit> jaraco: yes, I definitely agree extras should be supported, I was just joking that the obvious implementation of the functionality doesn't have an obvious way to check extras
[14:11:42] <elibrokeit> e.g. https://github.com/FFY00/python-build/issues/25 stumbled over this
[14:12:09] <FFY00> yes, I totally forgot to check for extra dependencies in my original implementation
[14:12:33] <elibrokeit> so the problem is really "my implementation doesn't handle extras, oops, hopefully that doesn't matter because if anything does use them, we're broken"
[14:13:01] <FFY00> https://paste.xinu.at/NawEO/
[14:13:24] <elibrokeit> of course one could implement extras too, but at this point the copy-pasted code just grows and grows, and eventually cries out for a proper, robust implementation, possibly in packaging
[14:13:43] <FFY00> I agree
[14:15:54] <FFY00> elibrokeit btw, .get_all() is documented https://docs.python.org/3.9/library/email.message.html#email.message.EmailMessage.get_all
[14:17:13] <elibrokeit> FFY00: yes, I know, I said that importlib.metadata documents that you should not depend on the EmailMessage interface, and should only assume metadata() supports dict methods, which get_all is not.
[14:17:39] <elibrokeit> So you end up assuming the implementation of metadata is an EmailMessage, which the doc says "don't do"
[14:17:54] <elibrokeit> > Technically, the returned distribution metadata object is an email.message.Message instance, but this is an implementation detail, and not part of the stable API. You should only use dictionary-like methods and syntax to access the metadata contents.
[14:21:53] <jaraco> I've only recently realized those docs probably need to be updated.
[14:22:10] <jaraco> If multi-value keys are needed (and they are), at least that part of the interface is necessary.
[14:23:20] <FFY00> jaraco, it is still available in .items(), but you need to filter them
[14:23:31] <jaraco> What the project needs is: A test capturing the need for multi-valued keys, a protocol describing the interface that 'metadata' guarantees to provide, and then an update to the docs to point to the protocol as the proper interface such that EmailMessage need not be mentioned.
[14:23:50] <FFY00> I would be +1 on declaring .get_all() or something similar as public api
[14:24:38] <jaraco> That sounds right to me ^
[14:25:05] <jaraco> Filtering is clumsy; agreed it's insufficient.
[14:25:39] <jaraco> s/insufficient/inadequate/
[14:26:10] <FFY00> yes, it does the job, but this is such a common need that we really should provide a proper interface
[14:26:18] <FFY00> :)
[14:45:21] <jaraco> FFY00: If you're willing, I'd be delighted if you would file an issue at gitlab.com/python-devs/importlib_metadata, include a problem description (motivation) and possibly the recommendation I made above as the approach. If you wish to take it further and provide a MR, that would also be most appreciated.
[14:46:19] <FFY00> will add to my todo, but I am already struggling to handle all my other stuff :P
[14:48:52] <elibrokeit> I'll be happy to submit an issue
[14:49:18] <elibrokeit> I'm also the one who pointed out the problem:
[14:49:18] <elibrokeit> [09:18:19 PM] <elibrokeit> FFY00: (also note that .get_all() is an undocumented implementation detail, and you're only ever supposed to use dictionary methods/syntax. So I'm genuinely unsure what the expected API for this is, especially for things which unlike requires-dist, don't have special properties...)
[14:51:39] <FFY00> go ahead
[15:01:25] <elibrokeit> https://gitlab.com/python-devs/importlib_metadata/-/issues/126
[15:15:11] <PSFSlack2> <deveshkusingh> thanks pradyunsg. I have PM
[15:15:25] <PSFSlack2> <deveshkusingh> *I have PM'd you