| EmilienM | crinkle: unless I'm doing wrong in the apt config, 'proposed' is broken | 01:05 |
|---|---|---|
| EmilienM | jamespage: fyi, keystone is broken when installing 'proposed' packages | 01:12 |
| EmilienM | I'm trying again, I'm probably doing something wrong | 01:13 |
| EmilienM | it seems ubuntu-cloud-keyring can't be found | 01:24 |
| EmilienM | so it sounds keystone can't be installed | 01:44 |
| EmilienM | jamespage: https://jenkins02.openstack.org/job/gate-puppet-openstack-integration-dsvm-trusty/38/consoleFull | 01:44 |
| EmilienM | crinkle: so I rebase integ patches and tempest run well: https://jenkins01.openstack.org/job/gate-puppet-openstack-integration-dsvm-centos7/51/console | 01:58 |
| EmilienM | that means, https://review.openstack.org/#/c/216831/ - https://review.openstack.org/#/c/216938 and https://review.openstack.org/#/c/217352/ can be reviewed | 01:59 |
| *** richm has quit IRC | 03:04 | |
| *** timrc has quit IRC | 03:31 | |
| *** timrc has joined #openstack-sprint | 03:33 | |
| *** mrmartin has joined #openstack-sprint | 04:50 | |
| jamespage | EmilienM, 2015-09-03 01:37:07.039 | WARNING: The following packages cannot be authenticated! | 07:09 |
| jamespage | would indicate ubuntu-cloud-keyring is not installed | 07:10 |
| jamespage | you have to install that first | 07:10 |
| jamespage | prior to adding the source and updating | 07:10 |
| EmilienM | jamespage: it is, look at the logs | 11:48 |
| EmilienM | jamespage: 2015-09-03 01:35:39.499 | + /usr/bin/apt-get -y install ubuntu-cloud-keyring | 11:48 |
| jamespage | EmilienM, hmm | 11:52 |
| jamespage | EmilienM, I'm not able to reproduce that - I can install OK from proposed without hitting unauthed packaged | 11:52 |
| EmilienM | wierd | 11:53 |
| jamespage | EmilienM, did you see that generally or just for keystone? if there is a signing issue, it would impact on the entire archive, not just a single package | 11:53 |
| EmilienM | weird | 11:53 |
| EmilienM | generally | 11:53 |
| EmilienM | it affects some python deps only, you see | 11:53 |
| jamespage | probably the ones that come from cloud-archive, rather than the main archive | 11:54 |
| jamespage | as the signing key is not the same | 11:54 |
| EmilienM | mhh | 11:56 |
| EmilienM | jamespage: I'll investigate more | 11:56 |
| EmilienM | jamespage: if I show you /etc/apt/trustdb.gpg maybe you can see if the package is good | 11:58 |
| EmilienM | the version is 2012.08.14 | 11:59 |
| EmilienM | looking at http://logs.openstack.org/18/219818/5/check/gate-puppet-openstack-integration-dsvm-trusty/4fd8263/logs/dpkg-l.txt.gz | 11:59 |
| EmilienM | bbl | 12:00 |
| *** mrmartin has quit IRC | 12:24 | |
| *** richm has joined #openstack-sprint | 12:40 | |
| EmilienM | good morning :) | 12:59 |
| clayton | morning :) | 13:09 |
| clayton | I'll be in and out today, I've got back to school stuff to ferry the kids to. That's why I couldn't make the in person sprint originally. | 13:13 |
| pabelanger | morning | 13:13 |
| pabelanger | clayton: nice, my daughter starts today too | 13:14 |
| clayton | I thought we were the only place that started this late in the year :) | 13:14 |
| clayton | We have open house stuff today at two different schools, school starts next Tuesday. | 13:15 |
| EmilienM | jamespage: so I confirm I still have an issue, see https://jenkins06.openstack.org/job/gate-puppet-openstack-integration-dsvm-trusty/47/console | 13:35 |
| EmilienM | jamespage: I run apt-get update, then install the keyring package, then run apt-get update and then it fails to install keystone | 13:35 |
| *** vinsh has quit IRC | 13:45 | |
| EmilienM | Hunner, _ody: I would like to help with documentation to get modules approved, how can I help? | 14:07 |
| EmilienM | so after some investigation, it seems openstack packages on trusty-proposed/liberty are not correctly signed or something's wrong with keyring - https://jenkins01.openstack.org/job/gate-puppet-openstack-integration-dsvm-trusty/59/console | 14:15 |
| pabelanger | Did people get a chance to review: https://review.openstack.org/#/c/215696? And experimental job for puppet-openstack-integraiton so we could experiment with nodepool sub nodes | 14:17 |
| *** mrmartin has joined #openstack-sprint | 14:19 | |
| *** vinsh has joined #openstack-sprint | 14:28 | |
| EmilienM | pabelanger: youre missing +A from infra anteaya ^ | 14:59 |
| EmilienM | I think it's safe to keep it experimental | 14:59 |
| EmilienM | and won't break anything but help you to move forward | 14:59 |
| *** vinsh has quit IRC | 15:00 | |
| pabelanger | EmilienM: Ya, just looking for comments in general. Adding it into experimental won't break anything | 15:10 |
| EmilienM | anteaya: pabelanger also has https://review.openstack.org/#/c/219866/ | 15:10 |
| EmilienM | anteaya: maybe you could review the patch | 15:10 |
| nibalizer | good morning everyone | 15:18 |
| *** vinsh_ has joined #openstack-sprint | 15:18 | |
| anteaya | morning nibalizer | 15:19 |
| EmilienM | nibalizer: o/ | 15:19 |
| mfisch | morning all | 15:32 |
| mwhahaha | morning | 15:33 |
| mfisch | mwhahaha: I'll have some time in a bit if you want to knock out those bugs | 15:43 |
| mwhahaha | i've got a fuel irc meeting in ~15, so might have to wait until 11 your time | 15:44 |
| mfisch | I have a meeting then | 15:45 |
| mfisch | today sucks | 15:45 |
| mfisch | let me ping you later | 15:46 |
| mwhahaha | k | 15:46 |
| mfisch | I'll see what I can look at now as time permits | 15:46 |
| Hunner | EmilienM: I added stuff to the module approval etherpad section | 15:58 |
| EmilienM | Hunner: thx will look | 15:58 |
| EmilienM | Hunner: does it means "cinder, glance, horizon, neutron" are going to be approved like they are now? | 16:01 |
| Hunner | Yep | 16:01 |
| Hunner | EmilienM: All the magic is in the spreadsheet | 16:01 |
| EmilienM | Hunner: nice | 16:01 |
| EmilienM | Hunner: but I don't see them approved (yet) on https://forge.puppetlabs.com/openstack | 16:02 |
| EmilienM | maybe I need to be patient :-) | 16:02 |
| EmilienM | crinkle: ubuntu issue I had last night was on canonical side, fixed by jamespage - so now, I'm rechecking all the patches, if jenkins now pass, you'll be able to +2 again :-) | 16:07 |
| Hunner | EmilienM: Yeah, approval is a DB query, so I figured I get them all done at once so I don't have to bug the forge team over and over :) | 16:08 |
| Hunner | nibalizer: _ody: Meghan was sick all night so I'm going to probably get very little done today (on purpose) | 16:08 |
| EmilienM | Hunner: awesome | 16:09 |
| crinkle | EmilienM: okay | 16:09 |
| EmilienM | crinkle: good morning! | 16:09 |
| crinkle | EmilienM: o/ | 16:09 |
| nibalizer | Hunner: ok | 16:10 |
| nibalizer | Hunner: you staying home or comming to puppetconf | 16:10 |
| _ody | Hunner: I have a dentist appointment at 11 so I am probably also not getting a much done until the afternoon | 16:10 |
| nibalizer | s/puppetconf/puppetlabs/ | 16:10 |
| _ody | nibalizer: I suspect we remote it today and show up at the office again to close out the sprint tomorrow. | 16:11 |
| Hunner | nibalizer: I'll be at puppetconf | 16:11 |
| Hunner | nibalizer: Oh, no I'm staying home to take care of meghan and kate | 16:12 |
| Hunner | But yeah, in tomorrow | 16:12 |
| EmilienM | anteaya: pabelanger updated the commit msg: https://review.openstack.org/#/c/215696/ | 16:13 |
| pabelanger | EmilienM: ack | 16:14 |
| nibalizer | _ody: okay | 16:14 |
| nibalizer | that works better for me anyways | 16:14 |
| EmilienM | pabelanger: it was not for you :P | 16:14 |
| anteaya | EmilienM: just wanting to hear from clarkb, yes | 16:15 |
| clarkb | ya in general it seems ok, need to not skim and review it properly efore I vote | 16:15 |
| clarkb | and still catching up on email this morning | 16:15 |
| EmilienM | richm: do you have patches about v3 that are not reviewed? I want to help | 16:17 |
| EmilienM | crinkle: I'm waiting for your feedback about doc thread, specially about my last reply | 16:20 |
| anteaya | clarkb: thank you | 16:21 |
| EmilienM | crinkle: also I rebased integ patches so you can see tempest is running: https://review.openstack.org/#/c/217352/ | 16:25 |
| EmilienM | jamespage: sounds better: https://jenkins02.openstack.org/job/gate-puppet-openstack-integration-dsvm-trusty/43/console | 16:44 |
| EmilienM | w00t | 16:44 |
| jamespage | EmilienM, awesome | 16:44 |
| clarkb | anteaya: pabelanger +2 | 16:51 |
| anteaya | clarkb: thank you | 16:57 |
| pabelanger | clarkb: perfect | 16:58 |
| EmilienM | jamespage: cinder is failing: http://logs.openstack.org/25/219825/2/check/gate-puppet-cinder-puppet-beaker-rspec-dsvm-trusty/4a038c5/console.html#_2015-09-03_16_54_44_801 | 17:04 |
| pabelanger | okay, running for some lunch | 17:04 |
| EmilienM | jamespage: designate too: http://logs.openstack.org/26/219826/2/check/gate-puppet-designate-puppet-beaker-rspec-dsvm-trusty/7429f04/console.html#_2015-09-03_16_55_01_259 | 17:04 |
| EmilienM | and horizon: http://logs.openstack.org/29/219829/2/check/gate-puppet-horizon-puppet-beaker-rspec-dsvm-trusty/793410a/console.html#_2015-09-03_16_55_18_171 | 17:05 |
| EmilienM | probably others... :( | 17:05 |
| EmilienM | only keystone works | 17:05 |
| jamespage | EmilienM, I may have to give you b2 to resolve these things | 17:07 |
| jamespage | I've been testing this afternoon | 17:07 |
| mfisch | mwhahaha: i'm avail in about 20m, you? | 17:10 |
| mwhahaha | yes | 17:11 |
| richm | EmilienM: I don't think so - you have either already reviewed them, or we're waiting on another patch based on feedback | 17:13 |
| mfisch | mwhahaha: okay ready? | 17:36 |
| mwhahaha | sure | 17:36 |
| mfisch | I want to start with vswitch | 17:36 |
| mfisch | https://bugs.launchpad.net/puppet-vswitch/+bug/1403480 | 17:36 |
| openstack | Launchpad bug 1403480 in puppet-vswitch "Catalog compilation failed sometimes" [High,Triaged] | 17:36 |
| mfisch | this looks like a redhat issue from what I'm seeing | 17:36 |
| mfisch | redhat specific I mean | 17:36 |
| mwhahaha | yup and once again has a fix elsewhere not upstreamed | 17:37 |
| mwhahaha | so may be trivial to fix | 17:38 |
| mfisch | do you have any way to test it? | 17:39 |
| mfisch | the fix is there | 17:39 |
| mwhahaha | i might be able to later | 17:40 |
| mwhahaha | i don't have a redhat env at the moment | 17:40 |
| mfisch | maybe the godaddy guys | 17:40 |
| mfisch | mwhahaha: maybe someone on fuel has an env they could tyr this fix on real quick | 17:41 |
| mwhahaha | let me see | 17:41 |
| mwhahaha | since all of our stuff is ubuntu these days i might have an old env somewhere | 17:42 |
| mfisch | mwhahaha: if you guys are ubuntu then let someone else takeit | 17:42 |
| mfisch | I've noted it in the list | 17:42 |
| mfisch | and I will ask some folks that use RH | 17:42 |
| mfisch | move on? | 17:42 |
| mwhahaha | we don't actually use the vswitch stuff | 17:42 |
| mwhahaha | we have our own thing | 17:42 |
| mfisch | k | 17:43 |
| mfisch | this may be short ;) | 17:43 |
| mwhahaha | which leads to some awkwardness in neutron | 17:43 |
| mwhahaha | :D | 17:43 |
| mfisch | https://bugs.launchpad.net/puppet-vswitch/+bug/1245973 | 17:43 |
| openstack | Launchpad bug 1245973 in puppet-vswitch "bridge down after reboot" [Undecided,Triaged] | 17:43 |
| mfisch | I dont think thats an issue TBH | 17:43 |
| mfisch | I've never seen it | 17:43 |
| mfisch | bridge always comes up after reboot | 17:43 |
| mfisch | I may just commnt to that affect and reassign | 17:43 |
| mwhahaha | sounds good | 17:43 |
| crinkle | EmilienM: re docs thread, i didn't see anything i needed to respond to, my addition was just "please keep this in mind", not arguing | 17:44 |
| crinkle | EmilienM: re 217352 looks like it's still failing | 17:44 |
| mfisch | mwhahaha: https://bugs.launchpad.net/puppet-ceilometer/+bug/1475551 | 17:45 |
| openstack | Launchpad bug 1475551 in puppet-ceilometer "fail with "Could not enable openstack-ceilometer-alarm-evaluator" " [Undecided,New] | 17:45 |
| mfisch | we dont use ceilometer | 17:45 |
| mwhahaha | we can, though it might be faster to kick it back to the reporter and ask for more logs as he thought it wasn't enabled but the issue is that it couldn't be started | 17:46 |
| mwhahaha | so without knowing their config it's hard to know what went wrong | 17:46 |
| mfisch | let me read more | 17:46 |
| mfisch | updated bug | 17:47 |
| mfisch | move on? | 17:48 |
| mwhahaha | sure | 17:48 |
| mfisch | https://bugs.launchpad.net/puppet-horizon/+bug/1481538 | 17:49 |
| openstack | Launchpad bug 1481538 in puppet-horizon "Updating webroot requires changes in _variables.scss" [Undecided,Confirmed] | 17:49 |
| mfisch | I went over this with our horizon expert | 17:49 |
| mfisch | I'm not sure we want to fix this | 17:51 |
| mfisch | rather than just ask for a backport | 17:51 |
| mwhahaha | sounds reasonable | 17:51 |
| mfisch | this is a legit bug though | 17:51 |
| mfisch | without it on ubuntu you get no icons | 17:51 |
| mwhahaha | but that seems to not be a puppet bug but a horizon issue | 17:52 |
| mwhahaha | as indicated by the fix | 17:52 |
| mfisch | it is | 17:52 |
| mfisch | and it was fixed upstream | 17:52 |
| mfisch | recently | 17:52 |
| mwhahaha | so might be something to just document and leave as wont fix | 17:53 |
| mfisch | yeah | 17:53 |
| mfisch | lets move on | 17:55 |
| mfisch | https://bugs.launchpad.net/puppet-horizon/+bug/1470993 | 17:56 |
| openstack | Launchpad bug 1470993 in puppet-horizon "if compress_static=true failed to apply horizon puppets" [Undecided,Confirmed] | 17:56 |
| mfisch | this is a bug for sure | 17:56 |
| mfisch | and also fixed in fuel | 17:56 |
| mfisch | we might need it upstreamed | 17:56 |
| mwhahaha | yes | 17:56 |
| mwhahaha | we just fixed the packaging, so might want to spend some time to investigate if we should fix it in puppet or push it to the packagers | 17:57 |
| mfisch | ok | 17:58 |
| mwhahaha | oh they might have fixed it | 17:58 |
| mwhahaha | the acceptance test references https://bugs.launchpad.net/ubuntu/+source/horizon/+bug/1424042 | 17:59 |
| openstack | Launchpad bug 1424042 in horizon (Ubuntu) "openstack-dashboard-ubuntu-theme fails to install" [High,Fix released] | 17:59 |
| mfisch | lookng | 17:59 |
| mfisch | so fixed upstream | 18:01 |
| mwhahaha | not sure when tho | 18:01 |
| mfisch | I'm confused | 18:02 |
| mfisch | fuel marked it fixed but I dont think theres a fuel code change | 18:02 |
| mwhahaha | we fixed the horizon package | 18:02 |
| mfisch | oh you guys pushed a fix to ubuntu | 18:02 |
| mfisch | ok | 18:02 |
| mwhahaha | kinda of, we fixed our version of the package. but it looks like jamespage updated the other bug to indicate that is now fixed upstream | 18:03 |
| mfisch | "Ubuntu cloud archive packages still hit this issue." | 18:03 |
| mwhahaha | as of today | 18:03 |
| mfisch | this isnt really a puppet bug is it? | 18:03 |
| mwhahaha | not really | 18:04 |
| mwhahaha | we could hack around it with puppet, but it's a package issue | 18:04 |
| mfisch | I agree | 18:04 |
| mfisch | I think I will un-list puppet on here | 18:04 |
| mfisch | and note | 18:04 |
| mwhahaha | k | 18:05 |
| mfisch | last bug is this | 18:06 |
| mfisch | https://bugs.launchpad.net/puppet-nova/+bug/1472740 | 18:06 |
| openstack | Launchpad bug 1472740 in puppet-keystone "RFE: Ability to add parameters to db sync commands" [Undecided,In progress] - Assigned to Mike Dorman (mdorman-m) | 18:06 |
| mfisch | I have it hear in case it needs discussion | 18:06 |
| mfisch | here | 18:07 |
| mfisch | I'm guessing Mike is just going to do this | 18:07 |
| mfisch | it does not affect me | 18:07 |
| mwhahaha | yea it looks like he was waiting on the db_sync stuff | 18:07 |
| mwhahaha | so i guess it was updating the db sync items to take a configurable set of configuration file names or something | 18:08 |
| mfisch | ok | 18:08 |
| mfisch | so nothing here to talk about | 18:08 |
| mfisch | I think we're done mwhahaha | 18:09 |
| mwhahaha | ok | 18:09 |
| mwhahaha | i'll take a look at that vswitch change | 18:09 |
| mwhahaha | it seems to make sense that it just needs a require line | 18:09 |
| mwhahaha | i'll see if i can figure out a way to reproduce it | 18:09 |
| mwhahaha | i figure it's an ordering of file inclusion issue | 18:10 |
| mfisch | thanks | 18:22 |
| EmilienM | crinkle: so, ubuntu is broken in b1 (proposed) | 18:35 |
| EmilienM | crinkle: ack for doc, anteaya: I think we can follow-up and make sure keeping in mind what crinkle said (our README is for using modules) and our proposal about using openstack.org is for developers | 18:36 |
| EmilienM | crinkle: here is what james told me in private: http://paste.openstack.org/show/qKiDGxlYodcYQ1wH407n/ | 18:36 |
| EmilienM | so now I'm very confused about ubuntu. Now we have to wait to see if b2 is working | 18:37 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/220224 | 18:47 |
| anteaya | EmilienM crinkle okay great, just wanted to ensure we all were in a happy place with creating published docs | 18:47 |
| anteaya | will start setting up those patches for review | 18:47 |
| EmilienM | glance is broken on centos https://github.com/openstack/glance/commit/484ef1b40b738c87adb203bba6107ddb4b04ff6e | 18:53 |
| EmilienM | new dependency is missing | 18:53 |
| EmilienM | crinkle: we are wondering about the repo name for the developer doc | 18:55 |
| EmilienM | I was wondering about puppet-openstack-doc or puppet-openstack-documentation | 18:55 |
| EmilienM | any thoughts here? | 18:55 |
| crinkle | EmilienM: what do the other projects use? | 19:03 |
| EmilienM | crinkle: their own git repo | 19:03 |
| EmilienM | tempest.git, neutron.git, etc | 19:03 |
| EmilienM | but we don't have one repo | 19:03 |
| crinkle | ¯\_(ツ)_/¯ | 19:03 |
| EmilienM | :) | 19:03 |
| EmilienM | if no feedback, I'll go for puppet-openstack-doc | 19:04 |
| EmilienM | anteaya: let's wait until tomorrow morning | 19:04 |
| EmilienM | so people can have time to read ML and give their opinion | 19:04 |
| EmilienM | anteaya: and tomorrow morning we take decision of the repo naming. | 19:04 |
| EmilienM | and eventually start the work | 19:04 |
| mwhahaha | fuel uses fuel-docs | 19:05 |
| mwhahaha | just as a comment on that | 19:05 |
| anteaya | EmilienM: as you wish | 19:06 |
| * anteaya fetches a pitcher from the rafters | 19:06 | |
| EmilienM | mwhahaha: yeah or puppet-docs | 19:07 |
| EmilienM | but I like want to steal 'puppet' because infra also use it | 19:09 |
| mwhahaha | the danger of puppet-openstack-doc(s) is that it might be thought as only docs for a puppet-openstack module, but it would go well with the puppet-openstack-specs repo out there. | 19:10 |
| mwhahaha | names are hard :( | 19:10 |
| EmilienM | we have puppet-openstack-specs, puppet-openstack-integration | 19:12 |
| EmilienM | let's use puppet-openstack-docs or something | 19:12 |
| EmilienM | I think nobody really cares | 19:13 |
| mwhahaha | that too | 19:13 |
| EmilienM | but to keep the topic open to our group, I'll let people until tomorrow morning to say something else | 19:13 |
| EmilienM | otherwise I'll ask anteaya to use puppet-openstack-docs | 19:13 |
| pabelanger | nibalizer: experimental job for puppet-unit running now, just waiting for a node | 19:13 |
| EmilienM | richm: is this patch in staled? https://review.openstack.org/#/c/178456/ like can we abandon it? | 19:14 |
| richm | EmilienM: If Keystone ever fixes the bug where endpoints created with v3 cannot be viewed with v2, or if we ever switch the entire world to never use v2 and only use v3, then the review is valid | 19:22 |
| EmilienM | lol | 19:22 |
| EmilienM | fair answer | 19:22 |
| richm | EmilienM: so I'm not really sure what to do with it | 19:23 |
| richm | is there some way to mark a review Future | 19:24 |
| EmilienM | richm: I'll keep it for now - Im just triaging patches | 19:24 |
| EmilienM | richm: no, but -1 WORKFLOW seems good here | 19:24 |
| richm | ah, ok | 19:24 |
| EmilienM | pabelanger: https://review.openstack.org/178026 | 19:28 |
| EmilienM | any core can review https://review.openstack.org/#/c/209290/ and https://review.openstack.org/#/c/209292/ ? they are bump patches to go in Liberty, for Heat and Ironic | 19:31 |
| EmilienM | also this patch to enable manila logs in beaker: https://review.openstack.org/#/c/213509/ | 19:33 |
| EmilienM | crinkle: see what james told me: http://paste.openstack.org/show/qKiDGxlYodcYQ1wH407n/ | 19:38 |
| EmilienM | I replied on your comment | 19:40 |
| crinkle | so we should be using staging or proposed? | 19:40 |
| crinkle | i thought long-term proposed was the preferred option | 19:40 |
| *** mrmartin has quit IRC | 19:47 | |
| EmilienM | crinkle: I think we are in a bad period now | 19:53 |
| EmilienM | crinkle: we should used 'proposed' which is today b1 which is broken | 19:53 |
| EmilienM | so we have 3 options | 19:53 |
| EmilienM | 1/ wait for b2, coming tonight or this week-end and switch to 'proposed', and hope it works | 19:54 |
| EmilienM | 2/ continue to use staging and keep reporting bugs every 2 minutes (and eventually burnout) | 19:54 |
| EmilienM | 3/ do nothing, wait until trusty-updates/liberty (which should be the stable repo at the end) | 19:55 |
| EmilienM | I vote for 1/ or 3/ | 19:55 |
| EmilienM | if we vote for 3/, I'll propose to set beaker jobs non voting for trusty so we can move forward and continue to land patches (still tested on RDO) | 19:55 |
| crinkle | I vote for 1, that seemed like the best option yesterday and waiting till tonight doesn't seem unreasonable to me | 19:56 |
| EmilienM | crinkle: even monday | 20:08 |
| EmilienM | crinkle: we are not in a hurry now | 20:08 |
| EmilienM | mgagne: I can't find your bug about api paste ini file provider which is useless | 20:14 |
| EmilienM | mgagne: if you have the URL sometimes | 20:14 |
| EmilienM | _ody: I dropped a comment about that on https://docs.google.com/spreadsheets/d/1Q8Db0jrv3dZQ8FbKhwpIjJRkI29Q571-FGs6VD_mVSQ/edit#gid=677531679 | 20:16 |
| EmilienM | _ody: for heat blocker | 20:16 |
| EmilienM | crinkle: I hoped during the sprint we could merge my patches on integration testing (3 patches) so we pabelanger could rebase his work to initiate multi node and I could move forward with other services without rebasing all the time - what do you think? | 20:51 |
| crinkle | EmilienM: I still think don't think it makes sense to merge patches for tests when the tests are failing and therefore not adding any value | 20:54 |
| crinkle | EmilienM: can we add depends-on patches in order to make progress? | 20:55 |
| EmilienM | crinkle: they're failing on trusty only | 20:55 |
| EmilienM | but well, yeah we can continue to make progress using depends-on & rebase.. it's just non blocking to merge them since they are non voting and the code won't change since it's packaging issues | 20:56 |
| EmilienM | crinkle: so you want to wait until ubuntu works? | 20:59 |
| crinkle | EmilienM: i don't care that much | 21:00 |
| crinkle | what's the list again? | 21:00 |
| EmilienM | Neutron: https://review.openstack.org/216831 | 21:01 |
| EmilienM | Nova: https://review.openstack.org/216938 | 21:01 |
| EmilienM | and Tempest run: https://review.openstack.org/217352 | 21:01 |
| EmilienM | crinkle: "Shouldn't we be doing this in puppet?" > we can't, this the bash commands are hacks to have floating IP | 21:07 |
| mgagne | EmilienM: https://bugs.launchpad.net/puppet-cinder/+bug/1482351 | 21:08 |
| openstack | Launchpad bug 1482351 in puppet-cinder "Keystonemiddleware configs should be in main configuration file" [Low,Confirmed] | 21:08 |
| EmilienM | and l3 agent needs to be restarted | 21:08 |
| EmilienM | mgagne: you rocks | 21:08 |
| EmilienM | _ody, Hunner https://bugs.launchpad.net/puppet-cinder/+bug/1482351 see that bug, and the unit tests that are missing is our modules should not be a blocker IMHO | 21:08 |
| EmilienM | crinkle: there is no way to run it with puppet now I guess | 21:09 |
| _ody | EmilienM: They aren't. Only things in red are blockers. | 21:09 |
| crinkle | so i thought our tests were supposed to 1) run puppet then 2) run tempest, if there's a bunch of bash configuration in the middle that doesn't seem like our puppet works | 21:09 |
| EmilienM | _ody: nice, sorry for the noise | 21:09 |
| EmilienM | crinkle: the bash code is an hack to make floating IP works in OpenStack Infra | 21:10 |
| EmilienM | I took the devstack way to fake br-ex | 21:10 |
| EmilienM | and being able to ping floating IP with a fake public network | 21:10 |
| EmilienM | I create br-ex in bash because puppet-vswitch has a bug where the bridge is created only after second run, there is a TODO about that in the code | 21:11 |
| crinkle | EmilienM: is this to make multinode work? or is this still for single node? | 21:11 |
| EmilienM | it's single node | 21:11 |
| EmilienM | for multi node, we will have to do more tricks | 21:11 |
| EmilienM | if we want floating IPs | 21:11 |
| EmilienM | devstack does the same | 21:11 |
| EmilienM | because we don't use real public network | 21:12 |
| EmilienM | we fake it | 21:12 |
| crinkle | EmilienM: okay, could you expand the comment to explain that a little more | 21:12 |
| EmilienM | except if RAX/HP can give up public network interface :o) | 21:12 |
| EmilienM | crinkle: ok sure thing | 21:12 |
| clarkb | EmilienM: you should just be running the devstack-gate function IMO | 21:13 |
| clarkb | don't copy paste it, source it | 21:13 |
| EmilienM | clarkb: I did not copy/paste code | 21:13 |
| EmilienM | clarkb: devstack code is not doing exactly the same thing but we can think at sourcing devstack scripts later | 21:14 |
| EmilienM | clarkb: the thing is, it's not a function iirc | 21:14 |
| clarkb | it is a function | 21:14 |
| clarkb | not devstack, devstack-gate | 21:14 |
| EmilienM | clarkb: ok, looking now | 21:16 |
| clarkb | you need an overlay l2 network to be able to ping from one host to the other and vice versa | 21:16 |
| clarkb | and tempest tests do that so kinda necessary | 21:16 |
| EmilienM | crinkle: replied, let me know if it's enough ( I'm looking now to source devstack-gate ) | 21:17 |
| EmilienM | clarkb: right | 21:17 |
| EmilienM | clarkb: ovs_gre_bridge ? | 21:19 |
| clarkb | EmilienM: yup | 21:19 |
| EmilienM | k | 21:19 |
| clarkb | EmilienM: it treats the primary node as the "switch" and every other subnode connects to the primary | 21:20 |
| EmilienM | clarkb: using the function saves me 2 lines of code, thanks :) | 21:21 |
| crinkle | EmilienM: that explanation sounds good, could you add it as a comment in the code? | 21:22 |
| EmilienM | crinkle: yes. I'm also addressing clarkb suggestion | 21:22 |
| crinkle | EmilienM: cool | 21:23 |
| clarkb | EmilienM: that and you don't have to debug that stuff | 21:24 |
| clarkb | because trust me its crazy to debug | 21:24 |
| clarkb | and the first time someone asks for ssh access to the test env to debug their own overlay setup is when I tell them to use the one that we have working :) | 21:26 |
| clarkb | so just skip ahead and use the one that works | 21:26 |
| crinkle | thanks clarkb :) | 21:26 |
| EmilienM | yeah thanks | 21:31 |
| EmilienM | clarkb: which example do you suggest to use for single node? br-ex? | 21:31 |
| clarkb | for single node you odn't need anything special, neutron should handle it all | 21:31 |
| EmilienM | or br-ex is for dvr | 21:31 |
| clarkb | no br-ex is a general neutron thing, its the bridge where floating IPs are assignged and how they get off host to the world | 21:32 |
| EmilienM | I'm trying ti figure ovs_gre_bridge arguments | 21:32 |
| clarkb | br-int is where all the private VM traffic flows | 21:32 |
| EmilienM | yeah, but in your code it's enabled if dvr is enabled | 21:32 |
| EmilienM | I have basic knowledge on neutron, got it | 21:32 |
| clarkb | EmilienM: oh right because in non dvr the primary node is the network node and is where tempest runs | 21:33 |
| EmilienM | k | 21:33 |
| clarkb | EmilienM: so if tempest and netowrk node are in the same place yo udon't need this | 21:33 |
| EmilienM | clarkb: I don't need this function? | 21:33 |
| clarkb | you only need it if you split those two, which we do in DVR because each host is a network node iwth floating IPs attached to br-ex | 21:33 |
| clarkb | EmilienM: if you put tempest and network on the same host you shouldn't | 21:33 |
| clarkb | because everything gets backhauled over br-int to the network node | 21:34 |
| EmilienM | I don't see how you can test floating IP without it | 21:34 |
| EmilienM | at least it failed to me | 21:34 |
| EmilienM | clarkb: you still need to manage floating IP (external) network | 21:34 |
| clarkb | EmilienM: if tempest is on the same host as the network node all the floating IPs are on the same machine as tempest | 21:34 |
| EmilienM | yes | 21:35 |
| EmilienM | but how do you route 200.0.0.0/24 for example | 21:35 |
| clarkb | EmilienM: only if those IPs are on a different machine than tempest (or whatever it is that is talking to them) | 21:35 |
| clarkb | EmilienM: we use 172.24.5.0/25 and all those IPs are on the same host as tempest when not using DVR | 21:35 |
| clarkb | EmilienM: so the route is bsaed on locally attached interfaces, linux knows to put the packets there | 21:35 |
| EmilienM | so I just create an external network with neutron api and that's it? | 21:36 |
| clarkb | EmilienM: yes, but only if tempest is on the same machine as your br-ex | 21:36 |
| EmilienM | clarkb: I'm running single node | 21:36 |
| EmilienM | all the time | 21:36 |
| clarkb | if br-ex is in a differnet location as in the case of DVR or in the care of splitting netowrk services apart from tempest then you need the overlay | 21:37 |
| EmilienM | but you told be I did not need br-ex | 21:37 |
| EmilienM | forget about DVR now | 21:37 |
| EmilienM | we will see later that thing :-) | 21:37 |
| clarkb | EmilienM: you don't need to make it, iirc neutron makes it | 21:37 |
| EmilienM | wow | 21:37 |
| EmilienM | why did it fail for me | 21:37 |
| clarkb | it definitely makes br-int | 21:37 |
| EmilienM | ok let's try again | 21:37 |
| clarkb | maybe it only makes one of the two? that would be funny | 21:37 |
| EmilienM | in l3 agent, you specify br-ex | 21:37 |
| EmilienM | for the external bridge | 21:38 |
| EmilienM | so I need to set br-int? | 21:38 |
| clarkb | no I am just using br-int as a thing that neutron definitely creates | 21:38 |
| clarkb | and I am pretty sure that devstack does not make br-ex either and neutron creates it | 21:38 |
| EmilienM | yeah but how you tell neutron l3 agent to use br-int | 21:38 |
| EmilienM | because by defeault it's using br-ex | 21:38 |
| EmilienM | br-int is managed by neutron agent, right | 21:39 |
| EmilienM | but l3 agent is using br-ex by default | 21:39 |
| EmilienM | afik | 21:39 |
| clarkb | by the l2 agent? its for private VM networking not related to l3 iirc | 21:39 |
| clarkb | yes l3 uses br-ex because thats where you get out to the world | 21:39 |
| EmilienM | clarkb: https://github.com/openstack/neutron/blob/master/etc/l3_agent.ini#L67 | 21:39 |
| clarkb | yes br-ex is the correct name... | 21:40 |
| clarkb | I am sayin neutron makes the bridge for you | 21:40 |
| clarkb | you shouldn't need to do anything special | 21:40 |
| EmilienM | but I want to test l3 | 21:40 |
| EmilienM | not private VM networking | 21:40 |
| clarkb | I know ... | 21:40 |
| EmilienM | ok, let's try.. | 21:41 |
| clarkb | so you make floating IPs on some range, say 172.24.5.0/24. Those become interfaces attached to br-ex on that machine | 21:41 |
| clarkb | bceause you have local interfaces for those IP addresses linux does not need any special route information, it says "I am directly attached ot that IP I will just deliver packets to it" | 21:41 |
| clarkb | now you may need forwarding to be neabled in your kernel/iptables but otherwise it should just work | 21:41 |
| clarkb | now, what that cannot do is talk off of the one host | 21:42 |
| EmilienM | clarkb: right but I think you need to create the interface 172.24.5.1 as the external network gateway | 21:42 |
| clarkb | only if you want to be able to reach those VMs from elsewhere | 21:42 |
| clarkb | which was my point about tempest being colocated | 21:42 |
| clarkb | if the only traffic is on one host you don't need anything | 21:42 |
| clarkb | if the traffic spans >1 host then you need the overlay or similar | 21:42 |
| EmilienM | ok this is what I missed ^^^^ | 21:42 |
| EmilienM | kk | 21:42 |
| * EmilienM tyring | 21:42 | |
| * EmilienM trying* | 21:42 | |
| EmilienM | pabelanger: am I wrong or dsvm-centos7 image have not been built for long? | 21:47 |
| EmilienM | I see 2015-09-03 18:02 in nodepool.o.o | 21:48 |
| clarkb | EmilienM: correct there is a bug with centos and puppet and systemd and iptables so the build sfail | 21:51 |
| EmilienM | ok good to know | 21:51 |
| EmilienM | I was wondering why infra script were not updated | 21:51 |
| clarkb | EmilienM: tl;dr is we attempt to use iptables-server but systemd doesn't relaod its config to know that iptables is avalid service after the package install | 21:51 |
| clarkb | then we try refreshing the iptables service and it fails whihc makes the buildfail | 21:51 |
| EmilienM | oops | 21:51 |
| EmilienM | I would be interested to know why it fails to refresh the service | 21:52 |
| clarkb | I am pretty sre thats a centos7 bug but I haven't had the time to make yet another account to file the bug | 21:52 |
| EmilienM | with journalctl | 21:52 |
| clarkb | EmilienM: because the ocnfig isn't loaded. | 21:52 |
| clarkb | EmilienM: so the packge installs and puts the unit file in /usr/lib/systemd/system but no systemctl daemon-reload is issued | 21:52 |
| EmilienM | I can ping someone from RH | 21:52 |
| clarkb | so systemctl start iptable sfails | 21:52 |
| EmilienM | pabelanger: do you know someone we can escalate? | 21:52 |
| EmilienM | otherwise I'll ask to RDO people to help | 21:53 |
| clarkb | EmilienM: this is specifically on hpcloud which uses upstream images, rax doesn't have the problem but they make their own images which may include iptables already | 21:53 |
| EmilienM | I see | 21:53 |
| EmilienM | we need to fix it upstream | 21:53 |
| clarkb | EmilienM: when I wa stesting if I did puppet || systemctl daemon-reload && puppet the second puppet run succeeds | 21:54 |
| EmilienM | this is not really clean, yeah we should fix it | 21:54 |
| EmilienM | mfisch, crinkle: I'm trying to help debugging also with https://review.openstack.org/#/c/218587/ (devstack way again) | 22:02 |
| EmilienM | see http://logs.openstack.org/87/218587/4/check/gate-puppet-openstack-integration-dsvm-trusty/94fa17c/logs/dstat.txt.gz | 22:02 |
| pabelanger | EmilienM: Ya, ianw should have looked at it | 22:11 |
| pabelanger | EmilienM: I hope to look at it when back to Ottawa | 22:11 |
| EmilienM | nibalizer: sorry, I lost the URL of openstackci dashboard you made, can you paste it again please? | 22:41 |
| EmilienM | found it http://spencerkrum.com/openstack/ci/# | 22:45 |
| *** jeblair has quit IRC | 22:59 | |
| nibalizer | EmilienM: woo | 23:08 |
| EmilienM | pabelanger: see workaround https://review.openstack.org/#/c/217352/40/run_tests.sh,cm | 23:08 |
| EmilienM | nibalizer: I use it, I like it | 23:09 |
| nibalizer | if you actually like it we should move it to infra or somehow wrap it up in one of the infra systems | 23:09 |
| nibalizer | I don't like running/providing serivces external to infra | 23:09 |
| EmilienM | nibalizer: I think I already told you I liked it a lot and I was told a specs was in process | 23:10 |
| nibalizer | hrm | 23:10 |
| nibalizer | well that was right after I made it | 23:10 |
| nibalizer | so if it still proves valuable after the initial excitement has passed then that's telling | 23:11 |
| clarkb | nibalizer: is that something that grafana/grafyaml can do for us? | 23:12 |
| clarkb | (I am unsure of grafanas capabilities) | 23:12 |
| nibalizer | likely! | 23:12 |
| pabelanger | Ya, I want to dive into it and see how it works nibalizer | 23:17 |
| EmilienM | see you later folks | 23:21 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!