14:00:30 <mattmceuen> #startmeeting airship 14:00:31 <openstack> Meeting started Tue Jul 30 14:00:30 2019 UTC and is due to finish in 60 minutes. The chair is mattmceuen. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:35 <openstack> The meeting name has been set to 'airship' 14:00:51 <mattmceuen> #topic Rollcall 14:01:02 <mattmceuen> Here's our agenda for today, let's give it a couple mins: https://etherpad.openstack.org/p/airship-meeting-2019-07-30 14:01:03 <dwalt> o/ hey all! 14:01:05 <nishantkr> o/ 14:01:08 <seaneagan> o/ 14:01:11 <mattmceuen> Hey fellas :D 14:01:17 <mattmceuen> welcome back seaneagan 14:01:28 <obravo> o/ 14:01:46 <lawrancejing> o/ 14:02:04 <ian-pittwood> o/ 14:02:29 <mattmceuen> #topic Announcements 14:02:42 <mattmceuen> Just one announcement from me today: Working Committee nomination extension and final day = today 14:02:50 <mattmceuen> So turns out I sent out a typo in an email 14:03:06 <mattmceuen> I'd meant to close nominations for the working committee last monday, but the email said next monday 14:03:13 <mattmceuen> different dates in different places 14:03:20 <mattmceuen> s/monday/tuesday/ 14:03:34 * mattmceuen needs more coffee, why did we change this meeting to be earlier :p 14:04:03 <mattmceuen> Anyway, nominations for the working committee will close TODAY at the advertised time 14:04:40 <mattmceuen> That has worked out very well, since a lot of folks have nominated during the extra week! 14:04:52 <mattmceuen> Very exciting, and thanks to those who have nominated 14:05:02 <mattmceuen> any questions / concerns on the nomination stuff? 14:05:48 <mattmceuen> or other announcements? 14:05:52 <dwalt> nothing from me. Thanks for the reminder 14:06:28 <mattmceuen> There are a few nominations pending +2s, please have no fear: the rule is that folks have nominated by the date, not that the patchset has merged 14:06:58 <mattmceuen> But we should have all necessary reviews from the election officials by then anyhow 14:07:20 <mattmceuen> #topic Dependency management in Airship python projects 14:07:32 <mattmceuen> ian-pittwood, I think this is yours! 14:07:43 <ian-pittwood> Yup, thanks mattmceuen 14:07:54 <ian-pittwood> I pre-scripted my wall of text so let me just paste that in here 14:08:00 <mattmceuen> lol 14:08:02 <ian-pittwood> So Alex Hughes and I were having trouble a while back with dependency conflicts between Pegleg and it's dependencies Deckhand, Promenade, and Shipyard client. This is because all of these projects were built at different times and were pinned to specific versions of each dependency. Promenade doesn't even define a requirements.txt for pip to use. To me it seems that since these packages are used as libraries for end user applicatio 14:08:02 <ian-pittwood> they probably shouldn't rigidly peg dependencies. I also understand that there is often a need to do so though. So to solve the problem, I tried implementing several different dependency managers like poetry, pip-compile/sync, and Pipenv. These packages modify or wrap pip to resolve package versions that best match the application being installed and its dependencies. I personally landed on Pipenv and tested it out by implementing 14:08:02 <ian-pittwood> into Spyglass. So far I am happy with the results, but I want to get some feedback though before I try to get the same functionality merged into Pegleg. For an example, here is the implementation of Pipenv for pegleg https://review.opendev.org/#/c/664970/ 14:08:48 <ian-pittwood> Sorry I know it's kind of long. Pretty much I'm looking to put a package manager similar to ruby gem into airship end-user applications. That's the tldr 14:08:55 * mattmceuen digesting :) 14:09:17 <ian-pittwood> Wanted to get some opinions so just throw out questions or comments whenever you're all ready 14:09:27 <mattmceuen> Thanks for doing this research 14:09:55 <mattmceuen> So one question, how do Pipenv et al compare to openstack project best practices? 14:10:18 <mattmceuen> Openstack projects pull from a common set of per-release dependency ranges as well, right? 14:10:40 <sthussey> I think pipenv is a good solution 14:10:46 <ian-pittwood> Is that list somewhere? I'm not sure if I was aware of that mattmceuen 14:11:10 <ian-pittwood> I think it is as well. It seems to be the direction that python is moving towards as it is made by pypa, the same group that makes pip 14:11:35 <sthussey> There was talk of moving to it several months ago, just no bandwidth to do so 14:12:16 <mattmceuen> Here's a reference to what I was fishing for - I am not an expert in it so the most valuable thing I can do is give it to you to add to your analysis ian-pittwood: https://pypi.org/project/openstack-requirements/ 14:12:34 <ian-pittwood> The major issue I found is that you pretty much can't use it on projects that are dependencies for other projects. So it's not a perfect solution. I think it's going to take a mix of implementing pipenv on user-facing projects and fixing/updating/loosening dependency pins on projects being consumed 14:13:55 <ian-pittwood> I'll take a look at that. It doesn't look familiar so I wouldn't think we're making use of the openstack-requirements now 14:13:58 <mattmceuen> Using it on user-facing projects would certainly be a big improvement 14:14:30 <mattmceuen> Yeah, I wasn't recommending using openstack-requirements, just seeing if they have anything for use to use in our approach w.r.t. scaled cross-project dep management 14:15:01 <sthussey> since airship is deprecated it may not be worth the effort, but likely the stuff being used via programmatic API should be split out from the parent project 14:15:26 <sthussey> So likely deckhand or promenade shouldn't be a dependency, rather deckhand-engine or promenade-foo should be 14:15:30 <mattmceuen> airship is not deprecated? 14:15:41 <sthussey> the python codebase 14:15:53 <portdirect> its very much still in active development 14:15:55 <mattmceuen> ok - to be clear that's not deprecated either 14:16:13 <mattmceuen> it'll be phased out over time but ++ portdirect's point 14:16:23 <portdirect> this convo is testament to that 14:16:33 <mattmceuen> reimplemented and/or swapped out over time, I should say 14:16:35 <ian-pittwood> I'll at least put it into practice on pegleg since that was where we were having the main issues. I think if we can just kind of standardize the way projects are doing dependencies that would be a step in the right directions. Like Promenade should have a requirements.txt not just requirements-frozen.txt 14:16:59 <mattmceuen> that would be great ian-pittwood 14:17:01 <sthussey> well, when version 2.0 isn't building on it that is something. 14:17:09 <sthussey> no, promenade shouldn't have a requirements.txt 14:17:23 <sthussey> dependency management in promenade is a deliberate pattern 14:17:43 <ian-pittwood> Shouldn't pegleg be able to know what dependencies promenade needs? 14:17:45 <mattmceuen> I mean, starting with pegleg. would the idea be to use that as a model to look at and roll out after implemented in pegleg? 14:17:58 <portdirect> i think the conversation has merit - these things should evolve with the developers working on them 14:18:21 <sthussey> https://opendev.org/airship/promenade/src/branch/master/requirements-direct.txt 14:18:39 <ian-pittwood> Sure. It's already in Spyglass as well if we need another model to look at. It is actually a pretty simple thing to implement. The major problem is solving VCS dependency conflicts across airship due to hard pins 14:18:52 <ian-pittwood> sthussey other projects can't see that during a pip install I don't think 14:19:12 <portdirect> ian-pittwood: i think you are correct there 14:19:16 <evrardjp> Just FYI airship has a fair different dna than openstack, so I didn't expect this to be a big problem, as each of the services can be made independent? 14:19:22 <sthussey> so moving that to a different filename is fine 14:19:58 <portdirect> evrardjp: unfortunately they cannot, as they rely upon each other 14:20:00 <sthussey> but a new requirements file is not 14:20:08 <portdirect> eg: https://opendev.org/airship/promenade/src/branch/master/requirements-direct.txt#L18 14:21:03 <ian-pittwood> portdirect yeah that line would be a problem. Pegleg already requires a completely different version of deckhand 14:21:35 <ian-pittwood> This is why dependency management is so difficult here. Can't really do loose pins with VCS reqs 14:21:44 <portdirect> this is where the openstack model was born from, im not advocating it - but this issue is hard 14:22:56 <ian-pittwood> I probably won't touch promenade for now then. Might be good that pegleg doesn't see the requirements so my install doesn't fail 14:24:15 <mattmceuen> ian-pittwood, as sthussey mentioned time & effort is probably the only blocker :) If you can start with pegleg, then we can roll out pipenv from there. 14:24:27 <mattmceuen> Does that sound good? 14:24:46 <ian-pittwood> Yup that sounds good to me 14:24:53 <mattmceuen> awesome 14:25:14 <mattmceuen> if there's nothing else on this topic, I'll move on: 14:25:33 <mattmceuen> #topic New project request: porthole 14:25:46 <openstackgerrit> Daniel Pawlik proposed airship/treasuremap master: Set namespace value directly in airskiff charts https://review.opendev.org/673535 14:25:55 <mattmceuen> Unfortunately I don't think diwakarthyagaraj was able to get in, he was having some connectivity issues 14:26:09 <mattmceuen> But I'll do my best to give a brief brief overview of the request in his absence 14:26:54 <mattmceuen> porthole is a project that has been incubating in att-comdev within github for a while 14:28:34 <mattmceuen> sorry, got pulled away 14:29:05 <mattmceuen> the project is intended to provide secure access to CLI utilities for various infrastructure within airship and osh 14:29:36 <mattmceuen> Basically, to enforce RBAC requirements upon CLIs such as etcdctl, psql, etc etc 14:30:34 <mattmceuen> As the domain knowledge involved in this project overlaps so much with airship, I think it makes sense to move the project into airship proper -- that will get better visibility into reviews, as well as better visibility for use of the project within the community 14:30:55 <mattmceuen> https://github.com/att-comdev/porthole 14:31:32 <mattmceuen> Do you guys have questions/thoughts/concerns? 14:33:01 <mattmceuen> In that case, I'll open it up for +1s or -1s to migrate in this project to the airship family! 14:33:06 <mattmceuen> For my part, +1 14:33:08 <portdirect> i think porthole would be a great addition to airship 14:33:19 <portdirect> it should not be a *required* project 14:33:42 <mattmceuen> ++ 14:33:57 <portdirect> but as an augment for orgs that have sophisticated operations and access policyies i think it fills an essential role 14:36:07 <mattmceuen> I take your comment as a +1 as well portdirect? 14:36:37 <nishantkr> +1 14:36:52 <dwalt> +1, I don't have any concerns, and I hope that this brings more attention to the project! 14:37:17 <michael-beaver> +1 14:37:51 <kskels> +1 14:37:55 <mattmceuen> I think we have consensus - thanks guys. I'll work with the porthole team to get the project (and the team!) integrated into airship 14:38:25 <portdirect> oh sorry, yes thats a +1 from me ;) 14:38:28 <mattmceuen> :D 14:38:41 <mattmceuen> #topic Review Requests 14:38:47 <mattmceuen> https://review.opendev.org/#/c/672688/ -- Tungsten Fabric-based aiab-tf site 14:38:47 <mattmceuen> https://review.opendev.org/#/c/672478/ vs https://review.opendev.org/#/c/672540/ 14:38:47 <mattmceuen> https://review.opendev.org/#/c/671575/ - Shipyard 14:39:33 <mattmceuen> obravo has added a site to treasuremap that is modeled after aiab, but which incorporates Tungsten Fabric. Very cool, I'll give that a test, sir :) 14:39:58 <mattmceuen> It currently relies on a patchset in openstack-helm for tf 14:40:40 <mattmceuen> dwalt, can you give some context on the "vs" ? 14:40:46 <mattmceuen> looks like we lost dpawlik 14:40:48 <obravo> Well , it`s not going after , it`s a replacement of default aiab 14:41:39 <dwalt> mattmceuen: sorry, I'm not sure I follow on the "vs" 14:41:54 <mattmceuen> obravo: `aiab` and `aiab-tf` will continue on as separate site definitions, right? and you can use one or the other depending on whether you want to use ovs or tf? 14:42:20 <mattmceuen> dwalt: ok I'll take a look into those today and try to understand the vs 14:42:38 <dwalt> Oh, my apologies 14:42:47 <dwalt> I see that you're referring to the changes above 14:43:23 <mattmceuen> no problem :) does it ring a bell now? 14:43:27 <obravo> you choose one of one of them , depending which networking you prefer . Both of them are on site layer 14:43:37 <mattmceuen> obravo: perfect 14:44:22 <dwalt> Yep! I didn't add this one, but for Airskiff we have conflicts with OSH on the version of Docker installed. i.e. OSH installs Docker already in the minikube gate script, so we don't need to install it 14:44:54 <dwalt> Daniel submitted a change to remove it, but we can probably go a step further and just let the OSH script install all required dependencies 14:45:17 <dwalt> But I'm also okay with Daniel merging his first or removing the script in his patch :) 14:45:21 <mattmceuen> got it 14:45:24 <dwalt> Mine was just a spike 14:45:31 <dwalt> to verify that it works 14:45:57 <mattmceuen> yeah, as long as the airskiff scriptage already has a dependency on the osh scriptage, might as well remove the duplicate package install 14:46:10 <mattmceuen> Agree with your point dwalt 14:46:33 <dwalt> great. It'd be nice to get this job operational again, so some eyes on it would be appreciated 14:46:43 <mattmceuen> agree 14:47:08 <mattmceuen> #topic Roundtable 14:47:15 <jamesgu> dwalt: nice job to simply the airskiff job 14:47:54 <dwalt> ty jamesgu. And thank you Daniel (who I don't think is online) for the help :) 14:48:27 <mattmceuen> we have a few mins left team, any other topics to bring up? 14:49:31 <mattmceuen> In that case: meeting adjourned! 14:49:51 <mattmceuen> thanks everyone - any more thoughts/questions, please bring them up here in the chat any time 14:49:56 <mattmceuen> have a great week 14:50:06 <mattmceuen> #endmeeting