15:00:20 #startmeeting third-party 15:00:27 Meeting started Mon Feb 27 15:00:20 2017 UTC and is due to finish in 60 minutes. The chair is lennyb_. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:28 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:31 The meeting name has been set to 'third_party' 15:00:35 Hello 15:00:36 o/ 15:00:46 how are you asselin? 15:01:35 i'm fine, yourself? 15:02:11 our CI is down for a few days due to #link refs/changes/80/437880/1 15:02:26 and we are upgrading our Ci from Ub14.04 to 16.04 15:02:55 why the upgrade? did -infra upgrade already? 15:03:16 o/ 15:03:24 #link https://review.openstack.org/#/c/437880 15:04:02 seems that devstack was not 'checked' on Ub14.04 and either I need to FORCE=yes or upgrade the CI servers 15:04:20 asselin, I am upgrading Mellanox Internal CI 15:04:37 hi mmedvede 15:04:50 we are using FORCE all the time, makes things easier 15:05:17 I haven't seen issues....maybe we have it pinned somewhere 15:05:27 mmedvede...yeah, but we need to check newer OS somewhen :) 15:06:19 mmedvede, maybe I will take your proposal and upgrade things slower 15:06:42 anything else you would like to bring up/discuss/complain ? 15:06:52 lennyb: we are using 16.04, FORCE by default just makes it easier in case we decide to use something non-standard. 15:07:11 last week was not a good week for third-party CIs :) 15:07:22 did some of you saw 'No hosts found to map to cell, exiting.' issue? 15:08:04 mmedvede, I have this week 'not a good' one 15:08:11 lennyb: hi, can't be related to placement api introduction from earlier ? is nova-compute running 15:08:20 mmedvede: not good indeed 15:08:57 only I have is a request to review https://review.openstack.org/#/c/263021/ 15:09:03 mptacekx, 'InternalError: Nova requires QEMU version 2.1.0 or greater.' :) 15:09:53 mptacekx, thanks. 15:10:09 thanks asselin. 15:11:05 anything else on those issues? 15:11:41 lennyb: we dont face that, due to nfv issues being tested we are using newer qemu 2.5+ 15:13:09 anything else to discuss? 15:14:07 we're currently fighting with 2 issues: 1) local.conf handling change when https://review.openstack.org/#/c/326585/ got merged (need to adapt on that) 2) race condition on setuptools with https://bugs.launchpad.net/devstack/+bug/1667545 - WA mentioned inside that ticket with pinning on tempest currently works well 15:14:07 Launchpad bug 1667545 in devstack "install_tempest failed at uninstalling appdirs 1.4.1" [Undecided,In progress] 15:15:49 For some reason our CI (IBM PowerKVM) was not affected by the appdirs bug 15:16:23 thanks mptaceks, we are facing this issue as well 15:16:48 we do not pin tempest, and we use rather standard dib build process similar to upstream infra 15:16:52 mmedvede: also upstream jenkins was not affected by that, no idea why. just found small discrepancy in logs, failing logs are building cmd2 wheels, working upstream not 15:17:47 mmedvede: can you please share the "standard dib build process way" ? 15:18:33 mmedvede: how to prevent "bad" setuptools to be deployed on system w/o patching tempest ? 15:19:07 mptacekx: by "standard" I mean the one that infra team uses for their ubuntu images, they have a bunch of dib elements. We mostly copied that 15:20:27 mptacekx: I can share more details if you're interested after the meeting. But it could be too involved just to fix appdirs problem 15:21:23 mmedvede: thanks, I would appreciate that. maybe link to the relevant part of elements from project-config repo or somewhere would be great 15:21:24 mptacekx, IMHO the bug should be fixed, since workarounds are not welcome 15:22:06 lennyb: thanks, I would cross-check and remove the WA. Out of curiosity still interested in the "elements way" 15:22:51 mptacekx: that's the problem, I do not know which part is relevant :) Infra-specific elements are here https://github.com/openstack-infra/project-config/tree/master/nodepool/elements 15:23:23 and they also have part of image update in puppet, which is whole another level 15:23:39 so just for working around appdir problem, it is not worth it 15:24:48 mmedvede: IMHO newer setuptools is not installed because of something on cloud-image but beacuse of it's availability in repo and by using pip -U install setuptools and not restricted in requirements like for tempest it simply comes up 15:25:56 mptacekx: fwiw, here is their image update log: http://nb01.openstack.org/dib.ubuntu-xenial.log 15:26:00 mmedvede: I also have some indications that wa is now not needed (also from lennyb few rows back), maybe it's simply gone 15:27:25 mmedved: thanks, there are installing setuptools-34.2.0-py2.py3-none-any.whl which should be also wrong as it's from 34.x.x line, no idea why they were not impacted 15:29:49 I have a nodepool question. is there a way to save /var/log/nodepool/image/image.log unique for each image? we have 3 images and logs are overqritten 15:30:32 lennyb, yes there is 15:30:51 there's a script that will do that...let me look it up 15:31:35 #link nodepool log config tool http://git.openstack.org/cgit/openstack-infra/system-config/tree/tools/nodepool_log_config.py 15:32:36 didn't make it to project-config....I don't remember why, but I there was a reason 15:32:47 asselin, thanks, I will check it 15:33:34 ok yeah...the script is configured to look at project-config by default: http://git.openstack.org/cgit/openstack-infra/system-config/tree/tools/nodepool_log_config.py#n33 15:33:35 asselin, do I need to run it once ? 15:34:09 yes 15:34:32 asselin, thanks, I will try it. 15:34:57 I don't recallthe details, but it does work and very useful to have each image in their own log file. 15:35:31 I wander why it's not the default behaviour 15:36:29 ++ probably waiting for someone to propose the patch.. I remember discussing this with jeblair and mordred a long time ago. 15:36:57 I think because it would require more complexity from nodepool code, more code. This way it is just logging config, left as an exercise to the user. 15:37:36 anyway, the script is simple enough, works, and new images aren't added frequently 15:37:57 thanks. any other issues to discuss? 15:39:52 in that case, thank you all for coming. I will end the meeting now 15:42:31 asselin, mmedvede, mptacekx thanks for showing up. 15:43:13 #endmeeting 15:44:34 lennyb: thanks 15:44:55 bot is asleep? :) 15:45:52 mmedvede, I've saw him flirting with a bot-girl :), probably got lucky 15:47:47 #endmeeting 15:50:59 lennyb: your nick has changed since you started the meeting 15:51:14 lennyb: try `/nick lenyb_` and then #endmeeting again 15:51:20 er, lennyb_ 15:51:35 thanks fungi 15:51:41 #endmeeting