Wednesday, 2016-02-17

*** hichihara has joined #openstack-infra00:01
clarkbI cut concurrency from 8 to 4 and tests only took 60% longer00:01
clarkbso we likely are running into some cpu contention00:02
clarkbjeblair: is it not also related to taking the derivative of a derviative? your axis change00:03
*** dims_ has quit IRC00:03
*** sflanigan has quit IRC00:04
*** jpr has quit IRC00:04
*** camunoz has quit IRC00:04
*** rbrndt_ has quit IRC00:04
*** pahuang has quit IRC00:05
jeblairclarkb: i feel like this approach is like telling the user what they want is wrong.  what i wanted to see was this: https://review.openstack.org/#/c/226114/4..5/nodepool/tests/test_nodepool.py00:06
clarkbI get that, but we aren't directly diffing them so get a different result00:06
jeblairclarkb: sorry, meant https://review.openstack.org/#/c/226114/3..5/nodepool/nodedb.py00:06
*** dchen has joined #openstack-infra00:07
clarkbthose results can be made to determine the same information about the changes though00:07
*** camunoz has joined #openstack-infra00:08
clarkbinterdiff between 3 and 5 would be + old code added | + new code added00:08
clarkbyou still see the change00:08
*** hashar has quit IRC00:08
jeblairclarkb: sometimes the user really is wrong? :)00:08
clarkbno I am not suggesting the user is wrong00:09
clarkbI am saying what the user wnats is really difficult there is an alternative that we can provide that remains correct and functional00:09
jeblair(it's okay, sometimes they are!)00:09
*** aorourke has quit IRC00:10
*** Jeffrey4l has joined #openstack-infra00:11
jeblairclarkb: right, i agree -- i think this is very difficult to do correctly.  personally, i'm trying to make it easier to review changes.  i'm not convinced that for me, personally, creating a new kind of first-order-derivative diff interface is the least-impact way to do that.00:12
jeblairclarkb: i really like to be able to see how a patchset evolves over time, particularly in response to comments00:12
*** hamzy has quit IRC00:12
*** aeng has quit IRC00:13
*** sabeen3 has quit IRC00:13
clarkbI don't like that the alternative is to make the submission criteria harder when enough people have difficulty with the current setup00:13
jeblairclarkb: so if we can't find a way to automatically factor out a rebase, then i'll probably keep making multi-patch sets.00:14
clarkbgerrit actively rejects pushes like the ones I make today too which is an extra layer of overhead00:14
jeblairclarkb: okay, so maybe we don't put it in the new user docs.  :)00:14
*** amitgandhinz has quit IRC00:14
jeblairi've only ever asked people that really knew the system to do it; and most of them were working on really long-running projects where having a clean and easy to follow development history was really important.00:15
jeblairclarkb: this is perhaps unusual today since you took over a 5 month old patch of mine00:15
clarkbmaybe I do the interdiff thing locally if I need to00:16
*** Sukhdev has quit IRC00:18
jeblairwhat's not clear to me is what's going to happen the next time i encounter this?  should i ask you to split?  is it my problem?  should i push the patchset myself?  should i just stop reviewing patches where i have lost the development train of thought?  all i know is that this is really important to me as a reviewer, and it seems like you feel it's a really onerous thing for me to ask of you as a developer.00:18
*** ianw has quit IRC00:19
openstackgerritMerged openstack/requirements: Updated from generate-constraints  https://review.openstack.org/27992600:19
*** Kennan has quit IRC00:19
clarkbjeblair: I think that considering this is how gerrit works it is onerous00:19
*** gyee_ has quit IRC00:19
jeblairi've lost half a working day to the issue already (not to mention the fact that i've already spent about 2 working days trying to come up with a solution to this on my own and failing).00:19
clarkbjeblair: you can review the base..current easily enough00:19
*** gyee has joined #openstack-infra00:20
jeblairclarkb: did you see the base..current for what you uploaded?00:20
clarkbjeblair: which is the other alternative00:20
jeblairclarkb: it had 5 months of development history in there00:20
clarkbjeblair: no it didn't that was ps1..current00:20
clarkbbase is HEAD^ of the current patchset00:20
*** salv-orlando has quit IRC00:20
clarkb(gerrit calls it base)00:20
jeblairclarkb: wait what?00:21
jeblairclarkb: what are you suggesting i review?00:21
*** Sukhdev has joined #openstack-infra00:21
*** hamzy has joined #openstack-infra00:21
*** ianw has joined #openstack-infra00:21
clarkbjeblair: essentially git show of the patchset00:21
clarkbwhich is gerrits default00:21
clarkbor the interdiff00:21
jeblairclarkb: yeah, i can't see the difference?00:21
clarkbthose are the two options that are reasonable00:21
clarkbthen you use the interdiff00:21
clarkbor ask the person that pushed the patchset?00:21
clarkbI am just saying in the generally case 10 minutes of patch wrangling (or more) is more than the cost of the alternatives00:22
jeblairclarkb: okay, so you're drawing the line -- i'm not to expect you to separate rebase and change patchsets00:22
jeblairi get it00:22
*** pahuang has joined #openstack-infra00:22
jeblaireven if it's my own change you're pushing on top of00:22
*** jeblair has quit IRC00:22
openstackgerritMerged openstack/diskimage-builder: yum-minimal: pre-install base packages  https://review.openstack.org/27886000:25
*** jeblair has joined #openstack-infra00:26
*** Kennan has joined #openstack-infra00:27
*** aorourke has joined #openstack-infra00:27
*** yuanying_ has joined #openstack-infra00:28
*** EricGonczer_ has joined #openstack-infra00:29
*** aeng has joined #openstack-infra00:30
*** yuanying has quit IRC00:31
*** gildub has joined #openstack-infra00:31
*** markusz has quit IRC00:32
*** dims has joined #openstack-infra00:33
*** sridhar_ram has quit IRC00:36
openstackgerritgreghaynes proposed openstack/diskimage-builder: Install docker for tests  https://review.openstack.org/27802100:36
*** ZZelle_ has quit IRC00:37
*** ccrouch has quit IRC00:37
*** ajmiller_ has joined #openstack-infra00:38
*** sflanigan has joined #openstack-infra00:40
*** sflanigan has joined #openstack-infra00:40
*** Sukhdev has quit IRC00:40
*** rkukura has quit IRC00:41
*** markusz has joined #openstack-infra00:41
*** ajmiller has quit IRC00:41
*** niska has quit IRC00:43
*** camunoz has quit IRC00:43
mgagneis there a way to re-run a post job in Zuul?00:47
clarkbmgagne: sort of :) you can reenqueue it direclty on the gearman bus00:47
clarkbmgagne: there is a utility to do that in zuul itself let me dig that up00:47
* mgagne is listening00:47
mgagnezuul client?00:47
mgagnezuul enqueue --trigger gerrit --pipeline check --project example_project --change 12345,1 ?00:48
clarkbmgagne: I want to say zuul client only knows about check and get00:48
clarkbbceause it is change oriented00:48
mgagnesure00:48
jeblairmgagne: enqueue-ref00:48
jeblairclarkb: ^00:48
jeblairfor the others00:48
clarkboh it grew that feature00:48
mgagnecool :D00:48
clarkbhttps://git.openstack.org/cgit/openstack-infra/zuul/tree/tools/trigger-job.py is the thing I was htinking of which is no longer necessary I guess00:48
*** yuanying_ has quit IRC00:49
*** yuanying has joined #openstack-infra00:49
mgagneinteresting, post didn't run from gerrit but it worked manually00:49
*** kzaitsev_mb has quit IRC00:49
mgagnewill need to investigate I guess :-/00:49
mgagnethanks for the hint!00:50
*** SumitNaiksatam has quit IRC00:50
openstackgerritMasayuki Igawa proposed openstack-infra/subunit2sql: Add unit test for api.update_run  https://review.openstack.org/27771700:51
*** kzaitsev_mb has joined #openstack-infra00:51
clarkbmgagne: maybe the ref matcher didn't match?00:53
clarkbin the pipeline configuration?00:53
mgagneI have yet to understand how to read Zuul logs and see why it didn't fire the job00:53
mgagnehmm00:53
mgagnelet me check00:53
mgagnesuper simplified version: http://paste.openstack.org/show/487207/00:55
mgagneref: ^(?!refs/).*$ is the one used in infra already so I guess it's ok00:55
mgagneand job fired in check and gate already with similar file filters00:55
*** niska has joined #openstack-infra00:56
*** yuanying has quit IRC00:59
*** camunoz has joined #openstack-infra01:00
*** harlowja_at_home has quit IRC01:01
fungiclarkb: trigger-job.py is still useful if you need to rerun one job without reenqueuing all jobs for that ref (for recent example, pypi-upload succeeds but announce-release fails: if you use enqueue-ref the pypi-upload will get rejected but just the announce-release can be retriggered with trigger-job.py)01:01
*** doug-fish has quit IRC01:01
clarkbfungi: gotcha01:01
fungibut in the general sense, no not so much any more01:01
openstackgerritMasayuki Igawa proposed openstack-infra/subunit2sql: Add unit test for api.update_run  https://review.openstack.org/27771701:01
mgagnegood to know01:01
* mgagne documents all troubleshooting/workarounds01:02
*** doug-fish has joined #openstack-infra01:02
*** bswartz has joined #openstack-infra01:03
*** mtanino has quit IRC01:04
*** eil397 has quit IRC01:04
*** yuanying has joined #openstack-infra01:06
mgagnefrom zuul/debug.log http://paste.openstack.org/show/487208/01:07
*** hamzy has quit IRC01:08
*** hamzy has joined #openstack-infra01:08
mgagneand from zuul.log: http://paste.openstack.org/show/487209/01:11
*** austin81 has left #openstack-infra01:15
*** roxanaghe has quit IRC01:16
*** gongysh has joined #openstack-infra01:17
*** bhunter_ has quit IRC01:17
*** apoorvad_ has joined #openstack-infra01:21
*** apoorvad has quit IRC01:21
*** mayurig has quit IRC01:21
*** wwwbukolaycom has quit IRC01:23
*** ekarlso has quit IRC01:30
*** Swami has quit IRC01:31
*** jpr has joined #openstack-infra01:34
*** aeng has quit IRC01:36
*** camunoz has quit IRC01:36
*** sdake has quit IRC01:36
*** pvaneck has quit IRC01:39
*** tiswanso has joined #openstack-infra01:39
*** tiswanso has quit IRC01:39
mgagneclarkb: restarted zuul just in case it didn't pick up my config, still no luck :-/01:39
*** tphummel has quit IRC01:40
*** tiswanso has joined #openstack-infra01:40
mgagnecould it be the "Changeish does not support dependencies" ?01:40
*** sam_wan has joined #openstack-infra01:42
clarkbmgagne: let me see01:42
mgagnebecause I don't get the same logs when manually triggered01:42
mgagnelet me know if you need more logs, I merged an other change01:42
clarkbmgagne: no that doesn't look like it01:43
clarkbmgagne: that message is just telling you a code path that is null for non changes was treated as such wich is correct01:44
*** doug-fish has quit IRC01:44
clarkbmgagne: http://paste.openstack.org/show/487210/ is the relevant portion of the logs I htink01:44
*** doug-fish has joined #openstack-infra01:44
*** ekarlso has joined #openstack-infra01:45
*** wolverin_ has joined #openstack-infra01:45
*** zhurong has joined #openstack-infra01:45
*** thorst has joined #openstack-infra01:45
mgagnecould it be the files filter not properly working in post pipeline?01:45
mgagnein fact, I'm not sure it's an exclusive filter or not01:46
*** thorst has quit IRC01:47
*** doug-fis_ has joined #openstack-infra01:48
*** kzaitsev_mb has quit IRC01:48
*** thorst has joined #openstack-infra01:48
mgagnelike post can't filter on filters due to the nature of the event01:48
clarkboh that could be it01:48
clarkblet me check what the event includes01:48
*** wolverineav has quit IRC01:48
*** camunoz has joined #openstack-infra01:49
*** aeng has joined #openstack-infra01:49
*** sputnik13 has quit IRC01:49
*** doug-fish has quit IRC01:49
clarkbmgagne: https://review.openstack.org/Documentation/cmd-stream-events.html#_ref_updated is the event so I think that is the case01:50
mgagneoh =(01:50
clarkbthat event is pretty barebones no file info01:50
mgagneok, will test without filters01:50
*** thorst has quit IRC01:52
*** doug-fis_ has quit IRC01:52
jeblairgreghaynes: i just wanted to let you know i read your zk comments this morning and am digesting them01:54
* mgagne read "disgusted by them" instead of "digesting them" (put popcorn away)01:55
*** zhurong has quit IRC01:55
mgagneclarkb: that was it! :D01:56
mgagneclarkb: post pipeline based on ref-updated event can't filter on files.01:56
*** zhurong has joined #openstack-infra01:56
clarkbmgagne: ya it can only do things based on the ocntent of the event itself01:57
mgagneI'm fine with that I guess01:57
openstackgerritMerged openstack/requirements: Bump python-openstackclient version to 2.1.0  https://review.openstack.org/27842801:59
*** chenli has joined #openstack-infra02:00
chenlihello, I'm tryting to install devstack , but I get an error :  http://paste.openstack.org/show/487211/02:00
chenliI tried  "from iso8601 import iso8601" in python by hand02:01
chenliit do works,02:01
chenlianyone know why this error happens ?02:01
lifelesschenli: you have a badly packaed system installed python-subunit02:04
chenlilifeless: o ? what that means ?02:04
lifelesspackaged, sorry.02:04
*** Sukhdev has joined #openstack-infra02:05
*** Sukhdev has quit IRC02:05
chenlilifeless: so, I should remove python-subunit ?02:05
lifelessworth a try02:06
*** yamamoto_ has joined #openstack-infra02:06
*** shashank_hegde has quit IRC02:08
*** zz_dimtruck is now known as dimtruck02:08
*** yamamoto_ has quit IRC02:09
*** Liuqing has joined #openstack-infra02:10
*** FallenPegasus has joined #openstack-infra02:11
*** FallenPegasus has quit IRC02:11
*** tqtran has quit IRC02:12
*** edmondsw has quit IRC02:12
*** zhurong has quit IRC02:19
*** zhurong has joined #openstack-infra02:20
chenlilifeless: it works ! Thanks !02:20
*** bpokorny_ has joined #openstack-infra02:24
*** changbl has joined #openstack-infra02:26
*** changbl has quit IRC02:26
*** markvoelker has quit IRC02:27
*** changzhi has joined #openstack-infra02:28
*** bpokorny has quit IRC02:28
*** bpokorny_ has quit IRC02:29
*** gangil has quit IRC02:31
*** _sarob has quit IRC02:31
*** apoorvad_ has quit IRC02:32
*** wolverin_ has quit IRC02:32
*** wolverineav has joined #openstack-infra02:33
*** EricGonczer_ has quit IRC02:37
*** doug-fish has joined #openstack-infra02:38
*** doug-fish has quit IRC02:39
openstackgerritSteve Martinelli proposed openstack/requirements: bump upper constraint for pycadf  https://review.openstack.org/28101502:39
*** doug-fish has joined #openstack-infra02:40
*** ajmiller_ has quit IRC02:44
*** kzaitsev_mb has joined #openstack-infra02:44
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: WIP Allow tempest-dsvm-full-{plugin} to set ds config  https://review.openstack.org/28101602:47
*** rhallisey has quit IRC02:50
*** changbl has joined #openstack-infra02:51
*** thorst has joined #openstack-infra02:51
*** doug-fish has quit IRC02:51
*** coolsvap|away is now known as coolsvap02:52
*** thorst has quit IRC02:57
*** rossella_s has quit IRC02:57
*** rossella_s has joined #openstack-infra02:58
*** salv-orlando has joined #openstack-infra02:59
openstackgerritxiangxinyong proposed openstack-infra/project-config: Add dashboard and client for Smaug  https://review.openstack.org/28101903:02
openstackgerritSteve Martinelli proposed openstack/requirements: bump upper constraint for keystoneauth  https://review.openstack.org/28102203:03
*** zhurong has quit IRC03:03
*** salv-orlando has quit IRC03:04
*** zhurong has joined #openstack-infra03:04
*** jpr has quit IRC03:07
openstackgerritSteve Martinelli proposed openstack/requirements: bump upper constraint for keystonemiddleware  https://review.openstack.org/28102803:08
*** yamamoto has joined #openstack-infra03:10
*** gyee has quit IRC03:11
*** ccrouch has joined #openstack-infra03:14
*** harlowja_at_home has joined #openstack-infra03:14
openstackgerritSteve Martinelli proposed openstack/requirements: bump upper constraint for keystoneclient  https://review.openstack.org/28103003:15
*** ccrouch1 has joined #openstack-infra03:16
*** ccrouch has quit IRC03:18
*** zhurong_ has joined #openstack-infra03:20
*** dimtruck is now known as zz_dimtruck03:20
*** SumitNaiksatam has joined #openstack-infra03:20
*** ccrouch1 has quit IRC03:21
*** zhurong has quit IRC03:21
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Add Neutron Failure Rate dashboard to Grafana  https://review.openstack.org/27883203:21
*** zz_dimtruck is now known as dimtruck03:22
*** dims has quit IRC03:23
*** markvoelker has joined #openstack-infra03:27
*** yuanying has quit IRC03:30
*** kzaitsev_mb has quit IRC03:31
*** yuanying has joined #openstack-infra03:32
*** changbl has quit IRC03:32
*** markvoelker has quit IRC03:33
*** aeng has quit IRC03:35
*** camunoz has quit IRC03:35
*** gangil has joined #openstack-infra03:36
*** gangil has joined #openstack-infra03:36
*** sflanigan has quit IRC03:38
*** harlowja_at_home has quit IRC03:40
*** changbl has joined #openstack-infra03:40
*** aeng has joined #openstack-infra03:47
*** camunoz has joined #openstack-infra03:48
*** dimtruck is now known as zz_dimtruck03:50
*** baoli_ has quit IRC03:50
*** sflanigan has joined #openstack-infra03:50
*** sflanigan has joined #openstack-infra03:50
*** shashank_hegde has joined #openstack-infra03:51
*** kzaitsev_mb has joined #openstack-infra03:51
*** baoli has joined #openstack-infra03:52
*** kushal has joined #openstack-infra03:52
*** baoli has quit IRC03:53
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add new job template for dsvm plugins with src  https://review.openstack.org/28101603:54
*** baoli has joined #openstack-infra03:54
*** kzaitsev_mb has quit IRC03:55
*** zhurong_ has quit IRC03:55
*** thorst has joined #openstack-infra03:56
*** gongysh has quit IRC04:00
*** zhurong has joined #openstack-infra04:00
*** fawadkhaliq has joined #openstack-infra04:02
*** thorst has quit IRC04:02
*** fawadkhaliq has quit IRC04:03
*** rlandy has quit IRC04:03
*** yuanying has quit IRC04:05
*** yuanying has joined #openstack-infra04:06
*** KanagarajM has joined #openstack-infra04:07
*** yuanying has quit IRC04:07
*** yuanying_ has joined #openstack-infra04:07
*** bpokorny has joined #openstack-infra04:10
*** zz_dimtruck is now known as dimtruck04:10
*** gangil has quit IRC04:12
*** zhurong has quit IRC04:13
*** zhurong has joined #openstack-infra04:14
*** chenli has quit IRC04:18
*** fedexo has joined #openstack-infra04:19
*** sflanigan has quit IRC04:20
*** camunoz has quit IRC04:20
*** aeng has quit IRC04:21
*** dchen has quit IRC04:21
*** chenli has joined #openstack-infra04:26
*** zhurong has quit IRC04:27
*** fawadkhaliq has joined #openstack-infra04:28
*** aeng has joined #openstack-infra04:31
*** sflanigan has joined #openstack-infra04:33
*** sflanigan has joined #openstack-infra04:33
*** fawadk has joined #openstack-infra04:34
*** zhurong has joined #openstack-infra04:34
*** fawadkhaliq has quit IRC04:35
*** dimtruck is now known as zz_dimtruck04:37
*** rguillebert has quit IRC04:37
*** ganso has quit IRC04:39
*** fawadk has quit IRC04:39
*** links has joined #openstack-infra04:41
openstackgerritMoshe Levi proposed openstack/diskimage-builder: Adding InfiniBand Support  https://review.openstack.org/27478104:43
greghaynesjeblair: awesome re: zk review - my hope is that would simplify the design a bit and let us break up the code too04:44
*** baoli has quit IRC04:44
jheskethAny eyes on https://review.openstack.org/#/c/281016/2 would be appreciated :-)04:51
*** zhurong has quit IRC04:55
*** links has quit IRC04:57
stevemarapparently zuul cloner is failing a lot04:58
*** sflanigan has quit IRC04:58
*** thorst has joined #openstack-infra04:59
*** bpokorny has quit IRC05:00
*** shashank_hegde has quit IRC05:03
*** zhurong has joined #openstack-infra05:04
*** sileht has quit IRC05:05
*** thorst has quit IRC05:06
openstackgerritCraig Vyvial proposed openstack-infra/project-config: run trove experimental run scenario tests  https://review.openstack.org/27809505:07
*** fawadkhaliq has joined #openstack-infra05:15
*** bexelbie has quit IRC05:16
*** zhurong has quit IRC05:16
*** zhurong has joined #openstack-infra05:19
*** sileht has joined #openstack-infra05:21
openstackgerritgreghaynes proposed openstack/diskimage-builder: Install docker for tests  https://review.openstack.org/27802105:21
*** bexelbie has joined #openstack-infra05:23
*** armax has quit IRC05:23
*** tiswanso has quit IRC05:24
*** shashank_hegde has joined #openstack-infra05:25
cp16netstevemar: yea i see that right now as well05:27
*** _sarob has joined #openstack-infra05:27
*** sdake has joined #openstack-infra05:27
*** kzaitsev_mb has joined #openstack-infra05:29
*** markvoelker has joined #openstack-infra05:29
*** camunoz has joined #openstack-infra05:30
*** gangil has joined #openstack-infra05:32
*** gangil has joined #openstack-infra05:32
*** _sarob has quit IRC05:32
*** markvoelker has quit IRC05:33
*** kzaitsev_mb has quit IRC05:34
*** gangil has quit IRC05:41
*** zhurong has quit IRC05:42
*** aukhan has joined #openstack-infra05:44
*** zhurong has joined #openstack-infra05:45
openstackgerritgreghaynes proposed openstack/diskimage-builder: Properly fail/trap in eval_run_d  https://review.openstack.org/25973405:46
*** Qiming has joined #openstack-infra05:47
*** zhurong has quit IRC05:47
*** _sarob has joined #openstack-infra05:48
*** zhurong has joined #openstack-infra05:48
*** sarob_ has joined #openstack-infra05:48
*** sdake has quit IRC05:49
*** Sukhdev has joined #openstack-infra05:51
*** jaosorior has joined #openstack-infra05:52
*** abregman has joined #openstack-infra05:52
*** _sarob has quit IRC05:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Fail functests if refusing to run tests  https://review.openstack.org/17700205:55
*** bexelbie has quit IRC05:57
*** bexelbie has joined #openstack-infra06:03
*** fedexo has quit IRC06:03
*** baoli has joined #openstack-infra06:04
*** thorst has joined #openstack-infra06:06
*** gildub has quit IRC06:06
*** gangil has joined #openstack-infra06:07
*** gangil has joined #openstack-infra06:07
*** sarob_ has quit IRC06:07
*** chenli has quit IRC06:09
*** baoli has quit IRC06:09
*** chenli has joined #openstack-infra06:10
*** Kennan has quit IRC06:10
*** Kennan has joined #openstack-infra06:11
*** thorst has quit IRC06:11
*** sflanigan has joined #openstack-infra06:13
*** sflanigan has joined #openstack-infra06:13
*** smanjara has joined #openstack-infra06:19
*** jaosorior has quit IRC06:19
*** jaosorior has joined #openstack-infra06:20
*** jaosorior has quit IRC06:20
*** jaosorior has joined #openstack-infra06:21
*** kushal has quit IRC06:26
*** wolverin_ has joined #openstack-infra06:26
*** fawadkhaliq has quit IRC06:27
*** fawadk has joined #openstack-infra06:27
*** _nadya_ has quit IRC06:27
*** _nadya_ has joined #openstack-infra06:28
*** tqtran has joined #openstack-infra06:29
*** wolverineav has quit IRC06:29
*** fawadk has quit IRC06:30
*** salv-orlando has joined #openstack-infra06:30
*** fawadkhaliq has joined #openstack-infra06:30
*** kzaitsev_mb has joined #openstack-infra06:31
*** sam_wan has quit IRC06:31
*** _nadya_ has quit IRC06:32
*** salv-orlando has quit IRC06:32
jeblairinfra-root: mirror.iad is not serving requests.  it looks like there were io errors on xvdb (a cinder volume) and the kernel remounted the afs cache dir read-only06:33
jeblairinfra-root: apparently the afs client did not deal with that gracefully06:33
*** e0ne has joined #openstack-infra06:33
jeblairinfra-root: i will attempt to repair (or replace) the fs and reboot06:33
*** kzaitsev_mb has quit IRC06:35
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/28107106:35
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/28107306:37
*** Nakato has quit IRC06:39
*** Nakato has joined #openstack-infra06:39
*** _nadya_ has joined #openstack-infra06:39
*** kushal has joined #openstack-infra06:41
*** tykeal has quit IRC06:42
*** wolverin_ has quit IRC06:42
*** gangil has quit IRC06:44
jeblairinfra-root: done; i disabled apache and openafs-client; rebooted to get a clean kernel state; unmounted the fs, ran fsck, it was clean, so i remounted and restarted the services.06:44
jeblaironce we're done with our mirror explosion, we may want to evaluate whether we can fit the afs cache on a local disk and avoid cinder on the mirror client-servers.06:45
openstackgerritxiangxinyong proposed openstack-infra/project-config: Add dashboard and client for Smaug  https://review.openstack.org/28101906:47
jeblair#status notice A problem with the mirror used for CI jobs in the rax-iad region has been corrected.  Please recheck changes that recently failed jobs on nodes in rax-iad.06:48
openstackstatusjeblair: sending notice06:48
-openstackstatus- NOTICE: A problem with the mirror used for CI jobs in the rax-iad region has been corrected. Please recheck changes that recently failed jobs on nodes in rax-iad.06:49
openstackstatusjeblair: finished sending notice06:50
Guest22736jeblair: thanks!06:55
*** Sukhdev has quit IRC06:56
*** rossella_s has quit IRC06:57
*** Guest22736 is now known as AJaeger06:58
*** rossella_s has joined #openstack-infra06:58
*** AJaeger is now known as Guest6074706:58
*** tykeal_away has joined #openstack-infra06:58
*** tykeal_away is now known as tykeal06:59
*** mikelk has joined #openstack-infra07:00
*** wolverineav has joined #openstack-infra07:01
*** Guest60747 has quit IRC07:03
*** AJaeger_ has joined #openstack-infra07:03
*** sam_wan has joined #openstack-infra07:04
*** maishsk has joined #openstack-infra07:04
stevemarjeblair: thanks :)07:05
*** maishsk has quit IRC07:06
*** thorst has joined #openstack-infra07:09
*** infra-red has joined #openstack-infra07:14
*** infra-re_ has joined #openstack-infra07:16
openstackgerritMerged openstack-infra/project-config: Use pep8 instead of linters for steth  https://review.openstack.org/27864407:17
*** thorst has quit IRC07:17
*** fawadk has joined #openstack-infra07:19
*** infra-red has quit IRC07:20
*** fawadkhaliq has quit IRC07:20
*** karlamrhein has joined #openstack-infra07:25
*** scheuran has joined #openstack-infra07:25
AJaeger_For later: https://github.com/openstack/giftwrap/ is still empty while http://git.openstack.org/cgit/openstack/giftwrap/ has content. Could any infra-root fix the github setup, please?07:26
*** julim has quit IRC07:28
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove now obsolete linters jobs  https://review.openstack.org/28109007:29
*** _nadya_ has quit IRC07:29
*** markvoelker has joined #openstack-infra07:30
*** ianw has quit IRC07:30
*** fawadkhaliq has joined #openstack-infra07:30
*** julim has joined #openstack-infra07:30
*** ianw has joined #openstack-infra07:30
*** fawadk has quit IRC07:30
*** maishsk has joined #openstack-infra07:31
*** kzaitsev_mb has joined #openstack-infra07:31
*** karlamrhein has quit IRC07:32
*** karlamrhein has joined #openstack-infra07:33
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add new job template for dsvm plugins with src  https://review.openstack.org/28101607:33
*** markvoelker has quit IRC07:34
*** jtomasek has joined #openstack-infra07:36
*** kzaitsev_mb has quit IRC07:36
*** rcernin has joined #openstack-infra07:42
*** ildikov has quit IRC07:46
*** maishsk has quit IRC07:49
openstackgerritMerged openstack-infra/infra-ansible: Create and attach other required openstack resources  https://review.openstack.org/28056107:50
*** maishsk has joined #openstack-infra07:51
openstackgerrityolanda.robla proposed openstack-infra/puppet-infracloud: Add element to bring up bridge on infra cloud  https://review.openstack.org/27629207:55
*** jlanoux has joined #openstack-infra07:56
yolandahi AJaeger, maybe manage-projects has not run again?07:56
yolandai'm going to check07:56
*** e0ne has quit IRC07:56
yolandaproject is created on gerrit07:57
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove check-deletions from openstack-manuals  https://review.openstack.org/28109707:58
yolandaAJaeger, i see http://git.openstack.org/cgit/openstack/giftwrap/tree/ with content, someone has already fixed?07:59
AJaeger_yolanda: or it failed setting up permissions07:59
AJaeger_yolanda: only github is broken AFAIK07:59
yolandaah07:59
yolandamaybe some replication problem, i need to check08:00
*** karlamrhein has quit IRC08:01
*** karlamrhein has joined #openstack-infra08:05
yolandaAJaeger, org.eclipse.jgit.errors.TransportException: git@github.com:openstack/giftwrap.git: push not permitted08:09
AJaeger_permissions?08:10
yolandalooks like08:10
AJaeger_Seems that the setup was not complete08:10
*** karlamrhein has quit IRC08:10
yolandaAJaeger, sorry but i'm not very familiar with github replications08:12
yolandai suppose that manage-projects creates the entry in github, then gerrit replicats as a normal target?08:12
AJaeger_yolanda: let's wait for others then, this is not urgent08:13
AJaeger_yolanda: thanks08:13
*** thorst has joined #openstack-infra08:14
*** ildikov has joined #openstack-infra08:16
*** karlamrhein has joined #openstack-infra08:16
yolandaAJaeger, if i follow documentation, and i do a "sudo su gerrit2" then "ssh github.com", i get permission denied08:16
yolandaso i wonder what has changed recently in terms of keys08:17
*** karlamrhein has quit IRC08:17
*** vincentfrancoise has joined #openstack-infra08:17
*** salv-orlando has joined #openstack-infra08:18
AJaeger_;(08:19
*** fawadkhaliq has quit IRC08:19
*** fawadkhaliq has joined #openstack-infra08:20
yolandagoing to check if i have the creds to enter into github ui08:20
*** salv-orlando has quit IRC08:20
openstackgerritVincent Françoise proposed openstack-infra/project-config: Watcher multinode Tempest job fixup  https://review.openstack.org/27792308:21
*** tqtran has quit IRC08:21
*** tzn has joined #openstack-infra08:22
*** thorst has quit IRC08:22
*** jpr has joined #openstack-infra08:22
*** bexelbie has quit IRC08:23
*** bexelbie has joined #openstack-infra08:23
AJaeger_another rebase just for the fun of it ^ ;(08:23
*** StevenK has quit IRC08:24
*** StevenK has joined #openstack-infra08:26
*** jpr has quit IRC08:26
yolandaAJaeger weird, i see ssh keys not touched in github since apr 2014, and last accessed today, but i cannot login into github from gerrit08:29
*** andymaier has joined #openstack-infra08:29
*** zeih_ has joined #openstack-infra08:30
AJaeger_fungi was able to fix a similar problem last week - I suggest you chat with him once he's online...08:30
yolandayes, i need to get that knowledge08:31
AJaeger_the repo is setup fine on openstack side, so the team can use it and thus this is good to fix but not urgent to learn it the hard way ;)08:31
*** kzaitsev_mb has joined #openstack-infra08:32
*** shardy has joined #openstack-infra08:35
*** kzaitsev_mb has quit IRC08:37
*** dizquierdo has joined #openstack-infra08:38
*** dingyichen has quit IRC08:41
*** pcaruana has joined #openstack-infra08:42
openstackgerritStefan Andres proposed openstack-infra/shade: Fix lookup for network parameter in neutron_available_floating_ips  https://review.openstack.org/28017808:42
*** matrohon has joined #openstack-infra08:43
*** Qiming_ has joined #openstack-infra08:44
*** dtardivel has joined #openstack-infra08:44
*** Qiming has quit IRC08:45
*** Qiming_ is now known as Qiming08:45
*** e0ne has joined #openstack-infra08:47
*** sputnik13 has joined #openstack-infra08:48
*** jpr has joined #openstack-infra08:50
*** sputnik13 has quit IRC08:53
*** zeih_ has quit IRC08:53
*** binbincong has quit IRC08:54
yolandaAJaeger, i'll ping fungi later about that. I checked logs and other repos are replicating fine to github. And i finally could ssh, i needed to use git user08:55
*** fhubik has joined #openstack-infra08:55
*** zeih_ has joined #openstack-infra08:55
yolandathe instructions on the system-config documentation don't specify that, i'll amend that bit08:55
rossella_sAJaeger_, hi Andreas!08:56
rossella_sAJaeger_, thanks a lot for your help08:56
*** zeih_ has quit IRC08:57
*** zeih_ has joined #openstack-infra08:57
*** Qiming has quit IRC08:57
*** Qiming has joined #openstack-infra08:57
openstackgerrityolanda.robla proposed openstack-infra/system-config: Specify git user when connecting to github  https://review.openstack.org/28112509:00
*** Qiming has quit IRC09:01
*** Qiming has joined #openstack-infra09:02
openstackgerritMerged openstack-infra/infra-manual: Fix typos and clarify project retirement  https://review.openstack.org/28082609:02
*** scheuran has quit IRC09:03
*** fhubik is now known as fhubik_brb09:05
openstackgerritMerged openstack-infra/project-config: run trove experimental run scenario tests  https://review.openstack.org/27809509:06
*** sshnaidm has quit IRC09:06
openstackgerritMerged openstack-infra/project-config: Add puppet-zaqar to tripleo jobs  https://review.openstack.org/28092809:07
openstackgerritMerged openstack-infra/infra-manual: Add URL for Gerrit group self-management  https://review.openstack.org/28082309:07
openstackgerritMerged openstack-infra/project-config: Fix name of ha-guide post job  https://review.openstack.org/28089909:07
*** fhubik_brb is now known as fhubik09:07
*** sshnaidm has joined #openstack-infra09:08
*** nelsnels_ has quit IRC09:09
openstackgerritJoshua Hesketh proposed openstack-infra/elastic-recheck: Add query for git repo not cloning 1542849  https://review.openstack.org/28113009:11
openstackgerritMerged openstack-infra/infra-manual: Update release management chapter  https://review.openstack.org/26061009:14
*** _nadya_ has joined #openstack-infra09:14
openstackgerritMerged openstack-infra/project-config: Remove unused periodic-pbr-installation-dsvm-upstream  https://review.openstack.org/28089409:15
AJaeger_rossella_s: please have a look whether that's a good basis. You can also send patches for reviewday to make it better suitable for neutron09:15
*** jmccrory_ has joined #openstack-infra09:16
*** jmccrory has quit IRC09:16
*** dizquierdo has quit IRC09:16
*** jmccrory_ is now known as jmccrory09:16
*** jmccrory has quit IRC09:16
*** achanda has quit IRC09:16
*** jmccrory has joined #openstack-infra09:17
*** nihilifer has quit IRC09:17
*** asettle has joined #openstack-infra09:17
*** AJaeger_ is now known as AJaeger09:18
rossella_sAJaeger_, thanks a lot! really...I am looking at it09:18
AJaeger;)09:18
openstackgerritMerged openstack-infra/project-config: Unblock refstack functional testing job  https://review.openstack.org/28040809:19
*** achanda has joined #openstack-infra09:19
*** binbincong has joined #openstack-infra09:20
*** thorst has joined #openstack-infra09:20
AJaegerrossella_s: credits go to clarkb, I just asked and relayed ;)09:20
*** zhurong has quit IRC09:21
*** nihilifer has joined #openstack-infra09:21
rossella_sthanks to both of you then AJaeger and clarkb09:22
*** [1]Thelo has joined #openstack-infra09:22
*** sfinucan has joined #openstack-infra09:24
*** Thelo has quit IRC09:25
*** [1]Thelo is now known as Thelo09:25
*** rvba has quit IRC09:26
*** rvba has joined #openstack-infra09:26
*** rvba has quit IRC09:26
*** rvba has joined #openstack-infra09:26
*** thorst has quit IRC09:27
openstackgerritMerged openstack-infra/project-config: Enable trove in devstack on stable/kilo  https://review.openstack.org/28046109:28
*** oanson has quit IRC09:28
*** karlamrhein has joined #openstack-infra09:28
*** oanson has joined #openstack-infra09:29
openstackgerritMerged openstack-infra/project-config: Adds neutron tempest tests for oslo.messaging drivers  https://review.openstack.org/27921509:30
openstackgerritMerged openstack-infra/project-config: Add check-requirements job  https://review.openstack.org/28055009:30
*** markvoelker has joined #openstack-infra09:31
openstackgerritMerged openstack-infra/project-config: Enable translation jobs for sahara-dashboard  https://review.openstack.org/28052009:31
openstackgerritMerged openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/23715609:31
*** kzaitsev_mb has joined #openstack-infra09:33
*** frickler has quit IRC09:34
*** max_lobur1 has quit IRC09:34
*** Mmike has quit IRC09:34
*** Sam-I-Am has quit IRC09:35
*** markvoelker has quit IRC09:35
*** Sam-I-Am has joined #openstack-infra09:35
*** karlamrhein has quit IRC09:35
*** chenli has quit IRC09:35
*** tdasilva has quit IRC09:38
*** kzaitsev_mb has quit IRC09:38
*** qb_ has joined #openstack-infra09:39
Kennanhi infra-cores, I found some jenkins jobs always failed with following info:09:39
Kennan provided hosts list is empty, only localhost is available09:39
Kennananyone knows that ?09:39
Kennanhttp://logs.openstack.org/44/281044/3/check/gate-functional-dsvm-magnum-api/9163b00/console.html09:40
*** achanda has quit IRC09:40
*** Qiming has quit IRC09:40
*** achanda has joined #openstack-infra09:40
*** Qiming has joined #openstack-infra09:41
*** Qiming_ has joined #openstack-infra09:41
*** Liuqing has quit IRC09:41
*** Mmike has joined #openstack-infra09:42
*** smanjara has quit IRC09:42
*** Qiming_ has left #openstack-infra09:42
*** derekh has joined #openstack-infra09:43
*** Mmike has quit IRC09:43
*** Mmike has joined #openstack-infra09:43
*** asselin__ has joined #openstack-infra09:44
*** shashank_hegde has quit IRC09:44
*** achanda has quit IRC09:46
*** Mmike has quit IRC09:47
*** asselin_ has quit IRC09:48
*** Mmike has joined #openstack-infra09:48
*** Mmike has joined #openstack-infra09:48
*** sputnik13 has joined #openstack-infra09:50
*** ganso has joined #openstack-infra09:50
*** karlamrhein has joined #openstack-infra09:51
*** max_lobur has joined #openstack-infra09:51
*** tdasilva has joined #openstack-infra09:51
*** Mmike has quit IRC09:52
*** ildikov has quit IRC09:52
*** teran_ has quit IRC09:52
*** ams_ has quit IRC09:52
*** pedroalvarez has quit IRC09:52
*** briancline has quit IRC09:52
*** tobiash_ has quit IRC09:52
*** bkero has quit IRC09:52
*** zz_ja has quit IRC09:52
*** edwarnicke has quit IRC09:52
*** zhenguo_ has quit IRC09:52
*** igorbelikov has quit IRC09:52
*** skath_ has quit IRC09:52
*** roaet_ has quit IRC09:52
*** HeOS__ has quit IRC09:52
*** ramishra has quit IRC09:52
*** bodepd_ has quit IRC09:52
*** mknawabi has quit IRC09:52
*** bryan_att has quit IRC09:52
*** sgordon has quit IRC09:52
*** markmcclain has quit IRC09:52
*** vipuls has quit IRC09:52
*** briancurtin has quit IRC09:52
*** pasquier-s has quit IRC09:52
*** bauzas has quit IRC09:52
*** madorn has quit IRC09:52
*** fabo has quit IRC09:52
*** lifeless has quit IRC09:52
*** _fortis has quit IRC09:52
*** GheRivero has quit IRC09:52
*** SotK has quit IRC09:52
*** hughhalf has quit IRC09:52
*** jraim has quit IRC09:52
*** serverascode has quit IRC09:52
*** dougwig has quit IRC09:52
*** sballe has quit IRC09:52
*** morgabra has quit IRC09:52
*** Odd_Bloke has quit IRC09:52
*** othermordred has quit IRC09:52
*** uberjay_ has quit IRC09:52
*** kzaitsev_ws has quit IRC09:52
*** JayF has quit IRC09:52
*** lane_kong has quit IRC09:52
*** zhiyan has quit IRC09:52
*** simonmcc has quit IRC09:52
*** jzerebecki has quit IRC09:52
*** tchaypo has quit IRC09:52
*** purp has quit IRC09:52
*** redrobot has quit IRC09:52
*** bookwar_ has quit IRC09:52
*** jkt has quit IRC09:52
*** rackertom has quit IRC09:52
*** petems has quit IRC09:52
*** etoews has quit IRC09:52
*** adreznec has quit IRC09:52
*** darvon has quit IRC09:52
*** DinaBelova has quit IRC09:52
*** swat30 has quit IRC09:52
*** skolekonov has quit IRC09:52
*** xgerman has quit IRC09:52
*** lindsayk_ has quit IRC09:52
*** patrickeast has quit IRC09:52
*** hrybacki has quit IRC09:52
*** warpc has quit IRC09:52
*** electrical has quit IRC09:52
*** reed has quit IRC09:52
*** Adri2000 has quit IRC09:52
*** amrith has quit IRC09:52
*** blogan has quit IRC09:52
*** jvrbanac has quit IRC09:52
*** afazekas has quit IRC09:52
*** mestery has quit IRC09:52
*** melwitt has quit IRC09:52
*** erikmwilson has quit IRC09:52
*** sulo has quit IRC09:52
*** briancline has joined #openstack-infra09:52
*** jzerebecki has joined #openstack-infra09:52
*** afazekas has joined #openstack-infra09:52
*** kzaitsev_ws has joined #openstack-infra09:52
*** purp has joined #openstack-infra09:52
*** karlamrhein has quit IRC09:52
*** redrobot has joined #openstack-infra09:52
*** sgordon has joined #openstack-infra09:52
*** jkt has joined #openstack-infra09:52
*** Odd_Bloke has joined #openstack-infra09:52
*** mknawabi has joined #openstack-infra09:52
*** amrith has joined #openstack-infra09:52
*** lifeless has joined #openstack-infra09:52
*** reed has joined #openstack-infra09:52
*** darvon has joined #openstack-infra09:52
*** Adri2000 has joined #openstack-infra09:52
*** Adri2000 has quit IRC09:52
*** Adri2000 has joined #openstack-infra09:52
*** jvrbanac has joined #openstack-infra09:52
*** uberjay has joined #openstack-infra09:52
*** pedroalvarez has joined #openstack-infra09:52
*** markmcclain has joined #openstack-infra09:52
*** HeOS_ has joined #openstack-infra09:52
*** fabo has joined #openstack-infra09:52
*** fabo has quit IRC09:52
*** fabo has joined #openstack-infra09:52
*** madorn has joined #openstack-infra09:53
*** SotK has joined #openstack-infra09:53
*** tchaypo has joined #openstack-infra09:53
*** bookwar has joined #openstack-infra09:53
*** blogan has joined #openstack-infra09:53
*** morgabra has joined #openstack-infra09:53
*** erikmwilson has joined #openstack-infra09:53
*** skolekonov has joined #openstack-infra09:53
*** redrobot is now known as Guest5316209:53
*** roaet_ has joined #openstack-infra09:53
*** pasquier-s has joined #openstack-infra09:53
*** vipuls has joined #openstack-infra09:53
*** bodepd has joined #openstack-infra09:53
*** hughhalf has joined #openstack-infra09:53
*** melwitt has joined #openstack-infra09:53
*** bmwiedemann has joined #openstack-infra09:53
*** sulo has joined #openstack-infra09:53
*** mestery has joined #openstack-infra09:53
*** bkero has joined #openstack-infra09:53
*** DinaBelova has joined #openstack-infra09:53
*** bauzas has joined #openstack-infra09:53
*** JayF has joined #openstack-infra09:53
*** tobiash has joined #openstack-infra09:53
*** swat30 has joined #openstack-infra09:53
*** arxcruz has quit IRC09:54
*** _fortis has joined #openstack-infra09:54
*** Mmike has joined #openstack-infra09:54
*** adreznec has joined #openstack-infra09:54
*** zz_ja has joined #openstack-infra09:55
*** frickler has joined #openstack-infra09:55
*** zeih_ has quit IRC09:55
*** smanjara has joined #openstack-infra09:56
*** Mmike has quit IRC09:56
*** Mmike has joined #openstack-infra09:56
*** tzn has quit IRC09:56
*** kragniz has quit IRC09:56
*** bmwiedemann has left #openstack-infra09:56
*** kragniz has joined #openstack-infra09:58
*** zhiyan has joined #openstack-infra09:58
*** lane_kong has joined #openstack-infra09:58
*** zhenguo_ has joined #openstack-infra09:59
*** warpc has joined #openstack-infra10:00
*** othermordred has joined #openstack-infra10:00
*** sballe has joined #openstack-infra10:00
AJaegerKennan: that is a warning. The problem is at10:00
*** lindsayk_ has joined #openstack-infra10:00
AJaegerhttp://logs.openstack.org/44/281044/3/check/gate-functional-dsvm-magnum-api/9163b00/console.html#_2016-02-17_08_39_05_73410:00
*** simonmcc has joined #openstack-infra10:00
*** GheRivero has joined #openstack-infra10:00
*** igorbelikov has joined #openstack-infra10:00
andreafAJaeger: https://review.openstack.org/#/c/278091/1 LGTM10:01
*** lucas-dinner has quit IRC10:01
*** xgerman has joined #openstack-infra10:01
*** ramishra has joined #openstack-infra10:01
*** electrical has joined #openstack-infra10:02
*** etoews has joined #openstack-infra10:02
*** jraim has joined #openstack-infra10:02
*** hrybacki has joined #openstack-infra10:02
*** serverascode has joined #openstack-infra10:02
*** jraim has quit IRC10:03
*** ramishra has quit IRC10:03
*** zhiyan has quit IRC10:03
*** sulo has quit IRC10:03
*** bauzas has quit IRC10:03
*** amrith has quit IRC10:03
*** jzerebecki has quit IRC10:03
*** darvon has quit IRC10:03
*** briancline has quit IRC10:03
*** tdasilva has quit IRC10:03
*** ganso has quit IRC10:03
*** jpr has quit IRC10:03
*** maishsk has quit IRC10:03
*** rcernin has quit IRC10:03
*** sam_wan has quit IRC10:03
*** AJaeger has quit IRC10:03
*** rossella_s has quit IRC10:03
*** bswartz has quit IRC10:03
*** niska has quit IRC10:03
*** jeblair has quit IRC10:03
*** vdrok has quit IRC10:03
*** nibalizer has quit IRC10:03
*** anthonyper has quit IRC10:03
*** johnthetubaguy has quit IRC10:03
*** jzerebecki has joined #openstack-infra10:03
*** jeblair has joined #openstack-infra10:03
*** nibalizer has joined #openstack-infra10:03
*** darvon has joined #openstack-infra10:03
*** scheuran has joined #openstack-infra10:03
*** amrith has joined #openstack-infra10:03
*** briancline has joined #openstack-infra10:03
*** AJaeger has joined #openstack-infra10:04
*** anthonyper has joined #openstack-infra10:04
*** niska has joined #openstack-infra10:04
*** dtantsur|afk is now known as dtantsur10:04
*** bmwiedemann has joined #openstack-infra10:04
*** rcernin has joined #openstack-infra10:04
*** briancurtin has joined #openstack-infra10:05
*** bauzas_ has joined #openstack-infra10:05
*** sulo has joined #openstack-infra10:05
*** maishsk has joined #openstack-infra10:05
*** johnthetubaguy has joined #openstack-infra10:05
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Cache is held and managed by connections  https://review.openstack.org/27639510:05
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Don't reconfigure connections on reload  https://review.openstack.org/28116610:05
*** rossella_s has joined #openstack-infra10:06
*** salv-orlando has joined #openstack-infra10:06
*** lucasagomes has joined #openstack-infra10:06
*** dougwig has joined #openstack-infra10:07
*** tdasilva has joined #openstack-infra10:07
AJaegerandreaf: thanks. The question by ianw was IMHO whether we need gate-tempest-dsvm-test-accounts-src-tempest-lib at all? Or could we revert your change completely? The job is still experimental...10:07
*** skath_ has joined #openstack-infra10:07
AJaegerandreaf: I'm fine with keeping it if it's needed10:08
*** mestery has quit IRC10:08
*** markmcclain has quit IRC10:08
*** asselin__ has quit IRC10:08
*** oanson has quit IRC10:08
*** infra-re_ has quit IRC10:08
*** camunoz has quit IRC10:08
*** sileht has quit IRC10:08
*** SumitNaiksatam has quit IRC10:08
*** yamamoto has quit IRC10:08
*** cgross- has quit IRC10:08
*** fbo_ has quit IRC10:08
*** tojuvone_ has quit IRC10:08
*** yamahata has quit IRC10:08
*** Anticimex has quit IRC10:08
*** vponomaryov has quit IRC10:08
*** imcsk8 has quit IRC10:08
*** chlong has quit IRC10:08
*** dkehn has quit IRC10:08
*** nijaba has quit IRC10:08
*** mordred has quit IRC10:08
*** bradm has quit IRC10:08
*** ajo has quit IRC10:08
*** charz_ has quit IRC10:08
*** jlvillal has quit IRC10:08
*** bradm has joined #openstack-infra10:08
*** mordred has joined #openstack-infra10:08
*** tojuvone_ has joined #openstack-infra10:08
*** Anticimex has joined #openstack-infra10:08
*** cgross has joined #openstack-infra10:08
*** markmcclain has joined #openstack-infra10:08
*** ajo has joined #openstack-infra10:08
*** fbo has joined #openstack-infra10:08
*** ams_ has joined #openstack-infra10:08
*** camunoz has joined #openstack-infra10:08
*** asselin__ has joined #openstack-infra10:08
*** teran_ has joined #openstack-infra10:08
*** mestery has joined #openstack-infra10:08
*** vponomaryov has joined #openstack-infra10:08
*** rackertom has joined #openstack-infra10:08
*** infra-red has joined #openstack-infra10:08
*** yamamoto has joined #openstack-infra10:08
*** sileht has joined #openstack-infra10:08
*** yamahata has joined #openstack-infra10:08
*** imcsk8 has joined #openstack-infra10:08
*** chlong has joined #openstack-infra10:09
*** bauzas_ is now known as bauzas10:09
*** charz has joined #openstack-infra10:09
*** jordanP has joined #openstack-infra10:10
*** oanson has joined #openstack-infra10:10
*** kushal has quit IRC10:12
*** ganso has joined #openstack-infra10:12
*** zhiyan has joined #openstack-infra10:12
*** ramishra has joined #openstack-infra10:12
*** wolverineav has quit IRC10:13
*** jraim has joined #openstack-infra10:13
*** petems has joined #openstack-infra10:13
*** tpsilva has joined #openstack-infra10:13
*** dkehn has joined #openstack-infra10:13
*** rguillebert has joined #openstack-infra10:13
*** jlvillal has joined #openstack-infra10:13
*** salv-orlando has quit IRC10:13
*** edwarnicke has joined #openstack-infra10:14
*** kzaitsev_mb has joined #openstack-infra10:14
*** bmwiedemann has left #openstack-infra10:14
*** vdrok has joined #openstack-infra10:14
*** jpr has joined #openstack-infra10:15
*** bryan_att has joined #openstack-infra10:15
*** Qiming has quit IRC10:15
*** patrickeast has joined #openstack-infra10:15
*** bauzas has left #openstack-infra10:17
*** vdrok has quit IRC10:17
arif-alihi all, how do we get the packages onto tarballs.openstack.org, we want to be able to get the salt-foumula-* added, so that these can be used for packaging10:18
*** sputnik13 has quit IRC10:18
*** andymaier has quit IRC10:18
*** kzaitsev_mb has quit IRC10:18
*** claudiub has joined #openstack-infra10:19
*** hichihara has quit IRC10:20
andreafAJaeger: yes I need that still - there are some race conditions in the test accounts code, and I need patches on top in tempest-lib to debug that - and I can only do that with a src job10:21
andreafAJaeger: I'm happy to drop the job completely once that is finished10:21
*** yamahata has quit IRC10:23
*** thorst has joined #openstack-infra10:24
*** jcoufal has joined #openstack-infra10:25
AJaegerandreaf: thanks - feel free to keep it as long as you need it ;)10:26
andreafAJaeger: nice, thanks10:26
AJaegeryolanda: so https://review.openstack.org/#/c/278091 is fine to approve  - as cleanup whenever you want10:30
yolandadone10:31
AJaegerSome projects - like manila - push artefacts there as part of the post job. See gate-manila-publishimage in project-config/jenkins/jobs/manila.yaml10:32
*** thorst has quit IRC10:32
*** salv-orlando has joined #openstack-infra10:32
AJaegerarif-ali: But it really depends on what exactly you want to push. If it's simply a tarball, use the tarball job...10:32
*** tzn has joined #openstack-infra10:32
AJaegerarif-ali: I'm busy with other things, can't help further right now10:33
AJaegerthanks, yolanda10:33
arif-aliAJaeger, no worries, I can wait, I am about to travel, But would like to understand how we get things onto the site10:35
*** andymaier has joined #openstack-infra10:36
*** tzn has quit IRC10:38
*** sambetts|afk is now known as sambetts10:39
*** jordanP has quit IRC10:39
*** jordan_ has joined #openstack-infra10:39
openstackgerritMerged openstack-infra/project-config: Remove tempest-dsvm-test-accounts-src from lib-forward-testing  https://review.openstack.org/27809110:39
AJaegerarif-ali: please write up briefly the workflow you have in mind and what you need for what - and then it's easier to discuss. Right now I'm missing lots of information to help further10:40
*** Liuqing has joined #openstack-infra10:40
*** electrofelix has joined #openstack-infra10:44
*** |-paul-| has joined #openstack-infra10:45
BobBallhttps://review.openstack.org/#/c/262661/ is failing on gate-nova-tox-functional for unrelated reasons but checking http://status.openstack.org/openstack-health/ to see how bad the job is, it doesn't seem to show the gate-nova-tox-functional job at all?10:48
*** rfolco has joined #openstack-infra10:48
*** dims has joined #openstack-infra10:49
*** salv-orlando has quit IRC10:49
*** patrickeast has quit IRC10:49
*** rackertom has quit IRC10:49
*** bradm has quit IRC10:49
*** niska has quit IRC10:49
*** scheuran has quit IRC10:49
*** jeblair has quit IRC10:49
*** nibalizer has quit IRC10:49
*** hrybacki has quit IRC10:49
*** serverascode has quit IRC10:49
*** GheRivero has quit IRC10:49
*** simonmcc has quit IRC10:49
*** zhenguo_ has quit IRC10:49
*** lane_kong has quit IRC10:49
*** uberjay has quit IRC10:49
*** pedroalvarez has quit IRC10:49
*** jvrbanac has quit IRC10:49
*** sgordon has quit IRC10:49
*** kzaitsev_ws has quit IRC10:49
*** derekh has quit IRC10:49
*** jmccrory has quit IRC10:49
*** jaosorior has quit IRC10:49
*** pahuang has quit IRC10:49
*** monester has quit IRC10:49
*** mkwiek has quit IRC10:49
*** gnuoy` has quit IRC10:49
*** flwang has quit IRC10:49
*** DuncanT has quit IRC10:49
*** ashtokolov has quit IRC10:49
*** erlon has quit IRC10:49
*** lbragstad_ has quit IRC10:49
*** lindsayk has quit IRC10:49
*** crinkle has quit IRC10:49
*** Zara has quit IRC10:49
*** cfarquhar has quit IRC10:49
*** hockeynut has quit IRC10:49
*** yolanda has quit IRC10:49
*** alex_xu has quit IRC10:49
*** dstufft has quit IRC10:49
*** pkoniszewski has quit IRC10:49
*** dolphm has quit IRC10:49
*** xek has quit IRC10:49
*** lmiccini has quit IRC10:49
*** logan- has quit IRC10:49
*** jokke_ has quit IRC10:49
*** calebb has quit IRC10:49
*** dmellado has quit IRC10:49
*** ianychoi has quit IRC10:49
*** mhayden has quit IRC10:49
*** khappone_ has quit IRC10:49
*** bmjen has quit IRC10:49
*** cloudnull has quit IRC10:49
*** d34dh0r53 has quit IRC10:49
*** lewo has quit IRC10:49
*** darrenc has quit IRC10:49
*** kuromagi^ has quit IRC10:49
*** Zara__ has quit IRC10:49
*** mescanef has quit IRC10:49
*** mikal has quit IRC10:49
*** whoops has quit IRC10:49
*** kaisers1 has quit IRC10:49
*** mmedvede has quit IRC10:49
*** vkramskikh has quit IRC10:49
*** cp16net has quit IRC10:49
*** clarkb has quit IRC10:49
*** leifmadsen has quit IRC10:49
*** odyssey4me has quit IRC10:49
*** eantyshev has quit IRC10:49
*** dtantsur has quit IRC10:49
*** alaski has quit IRC10:49
*** craige has quit IRC10:49
*** alaski has joined #openstack-infra10:49
*** gnuoy has joined #openstack-infra10:49
*** lbragstad_ has joined #openstack-infra10:49
*** jvrbanac has joined #openstack-infra10:49
*** pedroalvarez has joined #openstack-infra10:49
*** dolphm has joined #openstack-infra10:49
*** dtantsur has joined #openstack-infra10:49
*** clarkb has joined #openstack-infra10:49
*** mikal has joined #openstack-infra10:49
*** nibalizer has joined #openstack-infra10:49
*** jeblair has joined #openstack-infra10:49
*** xek has joined #openstack-infra10:49
*** bradm has joined #openstack-infra10:49
*** jokke_ has joined #openstack-infra10:49
*** ianychoi has joined #openstack-infra10:49
*** hockeynut has joined #openstack-infra10:49
*** uberjay has joined #openstack-infra10:49
*** cloudkiller has joined #openstack-infra10:49
*** yolanda has joined #openstack-infra10:49
*** jmccrory has joined #openstack-infra10:49
*** Zara has joined #openstack-infra10:49
*** calebb has joined #openstack-infra10:49
*** scheuran has joined #openstack-infra10:49
*** dmellado has joined #openstack-infra10:49
*** cp16net has joined #openstack-infra10:49
*** whoops has joined #openstack-infra10:49
*** kzaitsev_ws has joined #openstack-infra10:50
*** leifmadsen has joined #openstack-infra10:50
*** kaisers has joined #openstack-infra10:50
*** swat30 has quit IRC10:50
*** claudiub has quit IRC10:50
*** sgordon has joined #openstack-infra10:50
*** lewo has joined #openstack-infra10:50
*** alex_xu has joined #openstack-infra10:50
*** craige has joined #openstack-infra10:50
*** niska has joined #openstack-infra10:50
*** odyssey4me has joined #openstack-infra10:50
*** dstufft has joined #openstack-infra10:50
*** d34dh0r53 has joined #openstack-infra10:51
*** pkoniszewski has joined #openstack-infra10:51
*** kuromagi has joined #openstack-infra10:51
*** khappone has joined #openstack-infra10:51
*** lindsayk has joined #openstack-infra10:51
*** darrenc has joined #openstack-infra10:51
*** pahuang has joined #openstack-infra10:51
*** crinkle has joined #openstack-infra10:52
*** cfarquhar has joined #openstack-infra10:52
*** cfarquhar has quit IRC10:52
*** cfarquhar has joined #openstack-infra10:52
*** logan- has joined #openstack-infra10:52
*** phschwartz has quit IRC10:52
*** phschwartz_ has joined #openstack-infra10:52
*** ansiwen has quit IRC10:52
*** megm has quit IRC10:53
*** gnuoy_ has joined #openstack-infra10:53
*** jaosorior has joined #openstack-infra10:53
*** gnuoy_ has quit IRC10:53
*** cloudkiller is now known as cloudnull10:53
*** vkramskikh has joined #openstack-infra10:53
*** mhayden has joined #openstack-infra10:53
*** eantyshev has joined #openstack-infra10:54
*** fhubik has quit IRC10:54
*** bmjen has joined #openstack-infra10:54
*** dulek has quit IRC10:54
*** ansiwen has joined #openstack-infra10:54
*** dulek has joined #openstack-infra10:54
*** megm has joined #openstack-infra10:54
*** erlon has joined #openstack-infra10:55
*** Zara__ has joined #openstack-infra10:55
*** elo has quit IRC10:55
*** mescanef has joined #openstack-infra10:55
*** ashtokolov has joined #openstack-infra10:57
*** lmiccini has joined #openstack-infra10:57
*** GheRivero has joined #openstack-infra10:57
*** ldnunes has joined #openstack-infra10:57
*** rossella_s has quit IRC10:57
*** serverascode has joined #openstack-infra10:57
*** simonmcc has joined #openstack-infra10:57
*** aukhan has quit IRC10:58
*** tzn has joined #openstack-infra10:58
*** rossella_s has joined #openstack-infra10:58
*** andymaier has quit IRC10:58
*** lane_kong has joined #openstack-infra10:59
*** notnownikki has joined #openstack-infra11:00
*** monester has joined #openstack-infra11:00
*** patrickeast has joined #openstack-infra11:01
*** rackertom has joined #openstack-infra11:02
*** jlanoux has quit IRC11:02
*** DuncanT has joined #openstack-infra11:02
*** tzn has quit IRC11:03
openstackgerritAliaksandr Buhayeu proposed openstack/python-jenkins: Adds method to retrieve all jobs from the specific view  https://review.openstack.org/27975011:03
*** mkwiek_ has joined #openstack-infra11:03
*** rcernin has quit IRC11:03
*** zhenguo_ has joined #openstack-infra11:03
AJaegerBobBall: openstack-health handles only devstack jobs AFAIU11:03
*** mkwiek_ has quit IRC11:04
*** monester has quit IRC11:04
*** erlon has quit IRC11:04
*** vkramskikh has quit IRC11:04
*** darrenc has quit IRC11:04
*** kuromagi has quit IRC11:04
*** d34dh0r53 has quit IRC11:04
*** craige has quit IRC11:04
*** lewo has quit IRC11:04
*** whoops has quit IRC11:04
*** leifmadsen has quit IRC11:04
*** hockeynut has quit IRC11:04
*** cloudnull has quit IRC11:04
*** jokke_ has quit IRC11:04
*** bradm has quit IRC11:04
*** mikal has quit IRC11:04
*** dolphm has quit IRC11:04
*** dims has quit IRC11:04
*** edwarnicke has quit IRC11:04
*** ramishra has quit IRC11:04
*** zhiyan has quit IRC11:04
*** skath_ has quit IRC11:04
*** sulo has quit IRC11:04
*** briancurtin has quit IRC11:04
*** electrical has quit IRC11:04
*** xgerman has quit IRC11:04
*** igorbelikov has quit IRC11:04
*** sballe has quit IRC11:04
*** JayF has quit IRC11:04
*** blogan has quit IRC11:04
*** reed has quit IRC11:04
*** Adri2000 has quit IRC11:04
*** bkero has quit IRC11:04
*** Guest53162 has quit IRC11:04
*** Thelo has quit IRC11:04
*** asettle has quit IRC11:04
*** StevenK has quit IRC11:04
*** vincentfrancoise has quit IRC11:04
*** ameade has quit IRC11:04
*** clif_h has quit IRC11:04
*** Ng has quit IRC11:04
*** sweston has quit IRC11:04
*** mgkwill has quit IRC11:04
*** maximov has quit IRC11:04
*** wewe0901 has quit IRC11:04
*** peterlisak has quit IRC11:04
*** jed56 has quit IRC11:04
*** sbadia has quit IRC11:04
*** jswarren_ has quit IRC11:04
*** thomasem has quit IRC11:04
*** auggy has quit IRC11:04
*** mattymo has quit IRC11:04
*** hrubi has quit IRC11:04
*** xwizard_ has quit IRC11:04
*** HenryG has quit IRC11:04
*** betherly has quit IRC11:04
*** wolsen has quit IRC11:04
*** eliqiao_ has quit IRC11:04
*** mattoliverau has quit IRC11:04
*** med_ has quit IRC11:04
*** comstud has quit IRC11:04
*** rm_work has quit IRC11:04
*** mgagne has quit IRC11:04
*** zz_dimtruck has quit IRC11:04
*** dtroyer has quit IRC11:04
*** dkaigarodsev has quit IRC11:04
*** jbryce has quit IRC11:04
*** wendar has quit IRC11:04
*** beisner has quit IRC11:04
*** smcginnis has quit IRC11:04
*** cznewt has quit IRC11:04
*** timrc has quit IRC11:04
*** SlickNik has quit IRC11:04
*** mrodden has quit IRC11:04
*** Aegil has quit IRC11:04
*** mugsie has quit IRC11:04
*** Daviey has quit IRC11:04
*** gus has quit IRC11:04
*** elmiko has quit IRC11:04
*** tbarron has quit IRC11:04
*** ryanpetrello has quit IRC11:04
*** sigmavirus24_awa has quit IRC11:04
*** NikitaKonovalov has quit IRC11:04
*** lbragstad has quit IRC11:04
*** Guest40848 has quit IRC11:04
*** erikm has quit IRC11:04
*** lbragstad_ is now known as lbragstad11:04
*** matt6434 has joined #openstack-infra11:04
*** wendar has joined #openstack-infra11:04
*** jokke_ has joined #openstack-infra11:04
*** dolphm has joined #openstack-infra11:04
*** darrenc has joined #openstack-infra11:04
*** redrobot has joined #openstack-infra11:04
*** hrubi has joined #openstack-infra11:04
*** StevenK has joined #openstack-infra11:04
*** hockeynut has joined #openstack-infra11:04
*** med_ has joined #openstack-infra11:04
*** SlickNik has joined #openstack-infra11:04
*** NikitaKonovalov has joined #openstack-infra11:04
*** zz_dimtruck has joined #openstack-infra11:04
*** mgagne has joined #openstack-infra11:04
*** smcginnis has joined #openstack-infra11:04
*** Aegil has joined #openstack-infra11:04
*** beisner has joined #openstack-infra11:04
*** jbryce has joined #openstack-infra11:04
*** whoops has joined #openstack-infra11:04
*** blogan has joined #openstack-infra11:04
*** bradm has joined #openstack-infra11:04
*** asettle has joined #openstack-infra11:04
*** dims has joined #openstack-infra11:04
*** lewo has joined #openstack-infra11:04
*** Daviey has joined #openstack-infra11:04
*** redrobot is now known as Guest185611:04
*** med_ is now known as Guest550611:04
*** GheRivero has quit IRC11:04
*** dkaigarodsev has joined #openstack-infra11:04
*** craige has joined #openstack-infra11:04
*** thomasem has joined #openstack-infra11:04
*** mrodden has joined #openstack-infra11:05
*** sbadia has joined #openstack-infra11:05
*** zz_dimtruck is now known as dimtruck11:05
*** vincentfrancoise has joined #openstack-infra11:05
*** d34dh0r53 has joined #openstack-infra11:05
*** mugsie has joined #openstack-infra11:05
*** mattymo has joined #openstack-infra11:05
*** jswarren_ has joined #openstack-infra11:05
*** Guest40848 has joined #openstack-infra11:05
*** leifmadsen has joined #openstack-infra11:05
*** leifmadsen has quit IRC11:05
*** leifmadsen has joined #openstack-infra11:05
*** dtroyer has joined #openstack-infra11:05
*** peterlisak has joined #openstack-infra11:05
*** sulo has joined #openstack-infra11:05
*** xwizard_ has joined #openstack-infra11:05
*** JayF has joined #openstack-infra11:05
*** comstud has joined #openstack-infra11:05
*** HenryG has joined #openstack-infra11:05
*** wolsen has joined #openstack-infra11:05
*** tbarron has joined #openstack-infra11:05
*** ryanpetrello has joined #openstack-infra11:05
*** clif_h has joined #openstack-infra11:05
*** timrc has joined #openstack-infra11:05
*** kuromagi has joined #openstack-infra11:06
*** rm_work has joined #openstack-infra11:06
*** lbragstad_ has joined #openstack-infra11:06
*** sdague has joined #openstack-infra11:06
*** _fortis has quit IRC11:06
*** eliqiao_ has joined #openstack-infra11:06
*** bkero has joined #openstack-infra11:06
*** mmedvede has joined #openstack-infra11:07
*** vkramskikh has joined #openstack-infra11:07
*** cloudkiller has joined #openstack-infra11:07
*** Thelo has joined #openstack-infra11:07
*** gus has joined #openstack-infra11:07
*** reed has joined #openstack-infra11:07
*** pblaho has quit IRC11:07
*** sigmavirus24_awa has joined #openstack-infra11:08
*** gildub has joined #openstack-infra11:08
*** tzn has joined #openstack-infra11:08
*** cznewt has joined #openstack-infra11:09
*** mikal has joined #openstack-infra11:09
*** erikm has joined #openstack-infra11:09
*** elmiko has joined #openstack-infra11:09
*** Adri2000 has joined #openstack-infra11:09
*** hrybacki has joined #openstack-infra11:09
*** ramishra has joined #openstack-infra11:10
*** _fortis has joined #openstack-infra11:11
*** ameade has joined #openstack-infra11:11
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Cache is held and managed by connections  https://review.openstack.org/27639511:12
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Don't reconfigure connections on reload  https://review.openstack.org/28116611:12
*** betherly has joined #openstack-infra11:12
*** gildub has quit IRC11:12
*** auggy has joined #openstack-infra11:12
*** sileht has quit IRC11:13
*** wolverineav has joined #openstack-infra11:13
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/28107111:13
*** maximov has joined #openstack-infra11:14
*** erlon has joined #openstack-infra11:15
*** igorbelikov has joined #openstack-infra11:15
*** monester has joined #openstack-infra11:15
*** GheRivero has joined #openstack-infra11:15
*** sballe has joined #openstack-infra11:15
*** skath_ has joined #openstack-infra11:15
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: Add new job template for dsvm plugins with src  https://review.openstack.org/28101611:16
*** mkwiek_ has joined #openstack-infra11:16
*** Ng has joined #openstack-infra11:16
*** achanda has joined #openstack-infra11:16
*** zhiyan has joined #openstack-infra11:16
*** wewe0901 has joined #openstack-infra11:17
*** sileht has joined #openstack-infra11:17
*** mgkwill has joined #openstack-infra11:17
*** xgerman has joined #openstack-infra11:17
*** briancurtin has joined #openstack-infra11:17
*** wolverineav has quit IRC11:17
*** sweston has joined #openstack-infra11:18
*** jed56 has joined #openstack-infra11:18
*** aysyd has joined #openstack-infra11:18
*** kzaitsev_mb has joined #openstack-infra11:18
*** kushal has joined #openstack-infra11:19
*** electrical has joined #openstack-infra11:20
*** edwarnicke has joined #openstack-infra11:20
*** electrical has quit IRC11:21
*** msuriar has joined #openstack-infra11:21
*** achanda has quit IRC11:22
*** tzn has quit IRC11:22
*** tzn has joined #openstack-infra11:22
*** jlanoux has joined #openstack-infra11:28
*** cloudkiller is now known as cloudnull11:29
*** thorst has joined #openstack-infra11:30
*** markvoelker has joined #openstack-infra11:31
*** pblaho has joined #openstack-infra11:32
*** yamamoto has quit IRC11:32
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set validate_certs defaulting to no on all resources creation  https://review.openstack.org/28069711:35
*** changzhi has quit IRC11:35
*** markvoelker has quit IRC11:36
*** thorst has quit IRC11:36
*** reed has quit IRC11:39
*** Thelo has quit IRC11:39
*** lbragstad_ has quit IRC11:39
*** logan- has quit IRC11:39
*** crinkle has quit IRC11:39
*** |-paul-| has quit IRC11:39
*** dkehn has quit IRC11:39
*** imcsk8 has quit IRC11:39
*** frickler has quit IRC11:39
*** melwitt has quit IRC11:39
*** erikmwilson has quit IRC11:39
*** skolekonov has quit IRC11:39
*** SotK has quit IRC11:39
*** nihilifer has quit IRC11:39
*** mikelk has quit IRC11:39
*** sflanigan has quit IRC11:39
*** acabot has quit IRC11:39
*** Hunner has quit IRC11:39
*** Shrews has quit IRC11:39
*** cody-somerville has quit IRC11:39
*** krotscheck has quit IRC11:39
*** alazarev has quit IRC11:39
*** nt has quit IRC11:39
*** flip214 has quit IRC11:39
*** devananda has quit IRC11:39
*** mrunge has quit IRC11:39
*** markmcd1 has quit IRC11:39
*** markmcd has quit IRC11:39
*** spiffxp has quit IRC11:39
*** sabari has quit IRC11:39
*** ptoohill has quit IRC11:39
*** notmorgan has quit IRC11:39
*** Roamer` has quit IRC11:39
*** igormarnat has quit IRC11:39
*** asilenkov_ has quit IRC11:39
*** timfreund has quit IRC11:39
*** peterstac has quit IRC11:39
*** Hunner has joined #openstack-infra11:40
*** Hunner has quit IRC11:40
*** Hunner has joined #openstack-infra11:40
*** lbragstad_ has joined #openstack-infra11:40
*** Roamer` has joined #openstack-infra11:40
*** flip214 has joined #openstack-infra11:40
*** flip214 has quit IRC11:40
*** flip214 has joined #openstack-infra11:40
*** reed has joined #openstack-infra11:40
*** dkehn has joined #openstack-infra11:40
*** asilenkov_ has joined #openstack-infra11:40
*** nihilifer has joined #openstack-infra11:40
*** notmorgan has joined #openstack-infra11:40
*** notmorgan has quit IRC11:40
*** notmorgan has joined #openstack-infra11:40
*** mrunge has joined #openstack-infra11:40
*** frickler has joined #openstack-infra11:40
*** ptoohill has joined #openstack-infra11:40
*** skolekonov has joined #openstack-infra11:40
*** mikelk has joined #openstack-infra11:40
*** markmcd1 has joined #openstack-infra11:40
*** Shrews has joined #openstack-infra11:40
*** alazarev has joined #openstack-infra11:40
*** igormarnat has joined #openstack-infra11:40
*** peterstac has joined #openstack-infra11:40
*** imcsk8 has joined #openstack-infra11:40
*** melwitt has joined #openstack-infra11:40
*** erikmwilson has joined #openstack-infra11:40
*** |-paul-| has joined #openstack-infra11:40
*** devananda has joined #openstack-infra11:40
*** timfreund has joined #openstack-infra11:40
*** markmcd has joined #openstack-infra11:40
DuncanTHi. I've got a couple of tempest scenarios for cinder (main one does plenty of exercise of the quota code) that I don't want to run on the main gate. I plan on creating a gate job with a the fake backend driver eventually to allow this to be run in gate, but for now I just want to get it up and reviewed. Is there an established way of putting in non-gate11:40
DuncanTtests? (pointing me at an example and saying 'do it like this one' is fine)11:40
*** sflanigan has joined #openstack-infra11:40
*** sflanigan has joined #openstack-infra11:40
*** crinkle has joined #openstack-infra11:40
*** nt has joined #openstack-infra11:41
*** Thelo has joined #openstack-infra11:41
*** cody-somerville has joined #openstack-infra11:41
*** kushal has quit IRC11:41
*** acabot has joined #openstack-infra11:41
*** logan- has joined #openstack-infra11:42
*** spiffxp has joined #openstack-infra11:42
*** bauzas has joined #openstack-infra11:42
*** sabari has joined #openstack-infra11:42
*** krotscheck has joined #openstack-infra11:42
*** thomasem has quit IRC11:45
*** |-paul-| has quit IRC11:45
*** thomasem has joined #openstack-infra11:46
*** SotK has joined #openstack-infra11:46
openstackgerritMerged openstack-infra/infra-ansible: Set validate_certs defaulting to no on all resources creation  https://review.openstack.org/28069711:48
*** jamespag` is now known as jamespage11:48
*** jcoufal has quit IRC11:48
*** rcernin has joined #openstack-infra11:49
openstackgerritCostin Galan proposed openstack/requirements: Add Windows as supported operating system  https://review.openstack.org/27590511:49
*** spiffxp has quit IRC11:50
*** markmcd has quit IRC11:50
*** Shrews has quit IRC11:50
*** mikelk has quit IRC11:50
*** markmcd1 has quit IRC11:50
*** pblaho has quit IRC11:50
*** HenryG has quit IRC11:50
*** timrc has quit IRC11:50
*** SlickNik has quit IRC11:50
*** Zara__ has quit IRC11:50
*** ansiwen has quit IRC11:50
*** kaisers has quit IRC11:50
*** scheuran has quit IRC11:50
*** uberjay has quit IRC11:50
*** yolanda has quit IRC11:50
*** camunoz has quit IRC11:50
*** asselin__ has quit IRC11:50
*** jkt has quit IRC11:50
*** morgabra has quit IRC11:50
*** sfinucan has quit IRC11:50
*** _nadya_ has quit IRC11:50
*** jtomasek has quit IRC11:50
*** julim has quit IRC11:50
*** ianw has quit IRC11:50
*** Kennan has quit IRC11:50
*** gibi_ has quit IRC11:50
*** bapalm has quit IRC11:50
*** _diana_ has quit IRC11:50
*** toabctl has quit IRC11:50
*** BobBall has quit IRC11:50
*** ggillies has quit IRC11:50
*** stevebaker has quit IRC11:50
*** bnemec has quit IRC11:50
*** sheeprine has quit IRC11:50
*** jpeeler has quit IRC11:50
*** zeus has quit IRC11:50
*** jogo has quit IRC11:50
*** samueldmq has quit IRC11:50
*** NobodyCam has quit IRC11:50
*** lennyb_ has quit IRC11:50
*** katyafervent_awa has quit IRC11:50
*** pcrews has quit IRC11:50
*** abramley has quit IRC11:50
*** anteaya has quit IRC11:50
*** filler has quit IRC11:50
*** Swanson has quit IRC11:50
*** timothyb89 has quit IRC11:50
*** SotK_ has quit IRC11:50
*** tsufiev has quit IRC11:50
*** freerunner has quit IRC11:50
*** dteselkin has quit IRC11:50
*** kambiz has quit IRC11:50
*** schang has quit IRC11:50
*** ZZelle has quit IRC11:50
*** dangers_away has quit IRC11:50
*** notmyname has quit IRC11:50
*** mfisch has quit IRC11:50
*** matbu has quit IRC11:50
*** antonym has quit IRC11:50
*** rbradfor has quit IRC11:50
*** therve has quit IRC11:50
*** timburke has quit IRC11:50
*** gtmanfred has quit IRC11:50
*** grue_pm has quit IRC11:50
*** NobodyCam has joined #openstack-infra11:50
*** gibi has joined #openstack-infra11:50
*** uberjay has joined #openstack-infra11:50
*** sheeprine has joined #openstack-infra11:50
*** dangers_away has joined #openstack-infra11:50
*** ggillies has joined #openstack-infra11:50
*** stevebaker has joined #openstack-infra11:50
*** stevebaker has quit IRC11:50
*** stevebaker has joined #openstack-infra11:50
*** pblaho has joined #openstack-infra11:50
*** scheuran has joined #openstack-infra11:50
*** camunoz has joined #openstack-infra11:50
*** markmcd has joined #openstack-infra11:50
*** markmcd1 has joined #openstack-infra11:50
*** ZZelle has joined #openstack-infra11:50
*** SotK_ has joined #openstack-infra11:50
*** lennyb_ has joined #openstack-infra11:50
*** Swanson has joined #openstack-infra11:50
*** anteaya has joined #openstack-infra11:50
*** bapalm has joined #openstack-infra11:50
*** mikelk has joined #openstack-infra11:50
*** abramley has joined #openstack-infra11:50
*** pcrews has joined #openstack-infra11:50
*** _diana_ has joined #openstack-infra11:50
*** asselin__ has joined #openstack-infra11:50
*** BobBall has joined #openstack-infra11:50
*** sfinucan has joined #openstack-infra11:50
*** Shrews has joined #openstack-infra11:50
*** Kennan has joined #openstack-infra11:50
*** yolanda has joined #openstack-infra11:50
*** _nadya_ has joined #openstack-infra11:50
*** katyafervent_awa has joined #openstack-infra11:50
*** f1ller has joined #openstack-infra11:50
*** zeus has joined #openstack-infra11:50
*** freerunner has joined #openstack-infra11:50
*** timrc has joined #openstack-infra11:51
*** julim has joined #openstack-infra11:51
*** BobBall is now known as Guest1238311:51
*** antonym has joined #openstack-infra11:51
*** zeus is now known as Guest1077011:51
*** jtomasek has joined #openstack-infra11:51
*** rbradfor has joined #openstack-infra11:51
*** toabctl has joined #openstack-infra11:51
*** samueldmq has joined #openstack-infra11:51
*** notmyname has joined #openstack-infra11:51
*** schang has joined #openstack-infra11:51
*** HenryG has joined #openstack-infra11:51
*** SlickNik has joined #openstack-infra11:51
*** timothyb89 has joined #openstack-infra11:51
*** Daisy has joined #openstack-infra11:52
*** spiffxp has joined #openstack-infra11:52
*** morgabra has joined #openstack-infra11:52
*** Guest22953 has joined #openstack-infra11:53
*** timburke has joined #openstack-infra11:53
*** kaisers has joined #openstack-infra11:53
*** jpeeler has joined #openstack-infra11:53
*** jpeeler has joined #openstack-infra11:53
mordredyolanda: morning!11:54
mordredyolanda: anything you need me to look at?11:54
*** gtmanfred has joined #openstack-infra11:54
*** SotK_ has quit IRC11:54
yolandamordred, can you believe i was talking about you?11:54
yolandai am testing infra cloud east, and i'm getting errors, of nodepool trying to query for floating ips to neutron, even if i have floating_ip_source: 'None' on oscc11:55
*** grue_pm has joined #openstack-infra11:55
*** jkt has joined #openstack-infra11:55
*** ansiwen has joined #openstack-infra11:55
*** ianw has joined #openstack-infra11:56
mordredhrm11:56
mordredoh - yeah11:56
mordredyolanda: nodepool does not use shade for servers yet11:56
*** matbu has joined #openstack-infra11:56
*** Zara__ has joined #openstack-infra11:56
yolandamordred, as patches to review, i have https://review.openstack.org/266902, https://review.openstack.org/276292, https://review.openstack.org/280682, https://review.openstack.org/280720, https://review.openstack.org/280903 from my side, related to infra cloud11:57
yolandamordred, is there any way to tell nodepool to don't try generate floating ips?11:57
mordredwell - it only tries to generate floating ips if it can't figure out that your server got a public ip11:57
*** bnemec has joined #openstack-infra11:58
yolandamm, i have one of my servers with ip assigned, the other failed11:58
mordredyolanda: so I think there is a parameter you're missing until now about what network are it should use11:58
*** doug-fish has joined #openstack-infra11:58
mordredyolanda: or neutron is not configured such that the ips show up the way nodepool is expecting11:58
*** Qiming has joined #openstack-infra11:58
*** openstack has joined #openstack-infra12:03
*** openstackstatus has joined #openstack-infra12:03
*** asimov.freenode.net sets mode: -o trash12:03
-asimov.freenode.net- *** Notice -- TS for #openstack-infra changed from 1455710536 to 131551703212:03
*** asimov.freenode.net sets mode: +ct-s 12:03
*** mnaser has joined #openstack-infra12:03
*** chlong has joined #openstack-infra12:03
*** _fortis has joined #openstack-infra12:03
*** dims has joined #openstack-infra12:03
*** sirushti has joined #openstack-infra12:03
*** raginbajin has joined #openstack-infra12:03
*** SotK_ has joined #openstack-infra12:03
*** wolsen has joined #openstack-infra12:03
*** alex_xu has joined #openstack-infra12:03
*** jasondotstar has joined #openstack-infra12:03
*** salv-orlando has joined #openstack-infra12:03
*** greghaynes has joined #openstack-infra12:03
*** taron2 has joined #openstack-infra12:03
*** xwizard_ has joined #openstack-infra12:03
*** flaper87 has joined #openstack-infra12:03
*** SpamapS has joined #openstack-infra12:03
*** kevinbenton has joined #openstack-infra12:03
*** Guest45178 has joined #openstack-infra12:03
*** jd__ has joined #openstack-infra12:03
*** rhedlind has joined #openstack-infra12:03
*** fawadkhaliq has joined #openstack-infra12:03
*** pcaruana has joined #openstack-infra12:03
*** Roamer` has joined #openstack-infra12:03
*** jklare has joined #openstack-infra12:03
*** gothicmindfood has joined #openstack-infra12:03
*** nhandler has joined #openstack-infra12:03
*** Douhet has joined #openstack-infra12:03
*** SamYaple has joined #openstack-infra12:03
*** waynr has joined #openstack-infra12:03
*** aarefiev has joined #openstack-infra12:03
*** dolpher has joined #openstack-infra12:03
*** asettle has joined #openstack-infra12:03
*** mariojv has joined #openstack-infra12:03
*** cinerama has joined #openstack-infra12:03
*** fabo has joined #openstack-infra12:03
*** aviau has joined #openstack-infra12:03
*** olaph has joined #openstack-infra12:03
*** docaedo has joined #openstack-infra12:03
*** Qiming has joined #openstack-infra12:03
*** jeblair has joined #openstack-infra12:03
*** Guest40431 has joined #openstack-infra12:03
*** ctlaugh has joined #openstack-infra12:03
*** sflanigan has joined #openstack-infra12:03
*** cschwede has joined #openstack-infra12:03
*** jaime has joined #openstack-infra12:03
*** JayF has joined #openstack-infra12:03
*** dhellmann has joined #openstack-infra12:03
*** Guest46279 has joined #openstack-infra12:03
*** vsaienko has joined #openstack-infra12:03
*** bigjools has joined #openstack-infra12:03
*** stevelle has joined #openstack-infra12:03
*** mordred has joined #openstack-infra12:03
*** dosaboy has joined #openstack-infra12:03
*** lyz has joined #openstack-infra12:03
*** ttx has joined #openstack-infra12:03
*** GonZo2000 has joined #openstack-infra12:03
*** rvba has joined #openstack-infra12:03
*** mjb has joined #openstack-infra12:03
*** doug-fish has joined #openstack-infra12:03
*** bnemec has joined #openstack-infra12:03
*** Zara__ has joined #openstack-infra12:03
*** matbu has joined #openstack-infra12:03
*** ianw has joined #openstack-infra12:03
*** ansiwen has joined #openstack-infra12:03
*** jkt has joined #openstack-infra12:03
*** grue_pm has joined #openstack-infra12:03
*** gtmanfred has joined #openstack-infra12:03
*** jpeeler has joined #openstack-infra12:03
*** kaisers has joined #openstack-infra12:03
*** timburke has joined #openstack-infra12:03
*** Guest22953 has joined #openstack-infra12:03
*** morgabra has joined #openstack-infra12:03
*** spiffxp has joined #openstack-infra12:03
*** Daisy has joined #openstack-infra12:03
*** timothyb89 has joined #openstack-infra12:03
*** SlickNik has joined #openstack-infra12:03
*** HenryG has joined #openstack-infra12:03
*** schang has joined #openstack-infra12:03
*** notmyname has joined #openstack-infra12:03
*** samueldmq has joined #openstack-infra12:03
*** toabctl has joined #openstack-infra12:03
*** rbradfor has joined #openstack-infra12:03
*** jtomasek has joined #openstack-infra12:03
*** antonym has joined #openstack-infra12:03
*** julim has joined #openstack-infra12:03
*** timrc has joined #openstack-infra12:03
*** freerunner has joined #openstack-infra12:03
*** Guest10770 has joined #openstack-infra12:03
*** f1ller has joined #openstack-infra12:03
*** katyafervent_awa has joined #openstack-infra12:03
*** _nadya_ has joined #openstack-infra12:03
*** yolanda has joined #openstack-infra12:03
*** Kennan has joined #openstack-infra12:03
*** Shrews has joined #openstack-infra12:03
*** sfinucan has joined #openstack-infra12:03
*** Guest12383 has joined #openstack-infra12:03
*** asselin__ has joined #openstack-infra12:03
*** _diana_ has joined #openstack-infra12:03
*** pcrews has joined #openstack-infra12:03
*** abramley has joined #openstack-infra12:03
*** mikelk has joined #openstack-infra12:03
*** bapalm has joined #openstack-infra12:03
*** anteaya has joined #openstack-infra12:03
*** Swanson has joined #openstack-infra12:03
*** lennyb_ has joined #openstack-infra12:03
*** ZZelle has joined #openstack-infra12:03
*** markmcd1 has joined #openstack-infra12:03
*** markmcd has joined #openstack-infra12:03
*** camunoz has joined #openstack-infra12:03
*** scheuran has joined #openstack-infra12:03
*** pblaho has joined #openstack-infra12:03
*** stevebaker has joined #openstack-infra12:03
*** ggillies has joined #openstack-infra12:03
*** dangers_away has joined #openstack-infra12:03
*** sheeprine has joined #openstack-infra12:03
*** uberjay has joined #openstack-infra12:03
*** gibi has joined #openstack-infra12:03
*** NobodyCam has joined #openstack-infra12:03
*** rcernin has joined #openstack-infra12:03
*** SotK has joined #openstack-infra12:03
*** thomasem has joined #openstack-infra12:03
*** krotscheck has joined #openstack-infra12:03
*** sabari has joined #openstack-infra12:03
*** bauzas has joined #openstack-infra12:03
*** logan- has joined #openstack-infra12:03
*** acabot has joined #openstack-infra12:03
*** cody-somerville has joined #openstack-infra12:03
*** Thelo has joined #openstack-infra12:03
*** nt has joined #openstack-infra12:03
*** crinkle has joined #openstack-infra12:03
*** timfreund has joined #openstack-infra12:03
*** devananda has joined #openstack-infra12:03
*** erikmwilson has joined #openstack-infra12:03
*** melwitt has joined #openstack-infra12:03
*** imcsk8 has joined #openstack-infra12:03
*** peterstac has joined #openstack-infra12:03
*** igormarnat has joined #openstack-infra12:03
*** alazarev has joined #openstack-infra12:03
*** skolekonov has joined #openstack-infra12:03
*** ptoohill has joined #openstack-infra12:03
*** frickler has joined #openstack-infra12:03
*** mrunge has joined #openstack-infra12:03
*** notmorgan has joined #openstack-infra12:03
*** nihilifer has joined #openstack-infra12:03
*** asilenkov_ has joined #openstack-infra12:03
*** dkehn has joined #openstack-infra12:03
*** reed has joined #openstack-infra12:03
*** lbragstad_ has joined #openstack-infra12:03
*** Hunner has joined #openstack-infra12:03
*** jlanoux has joined #openstack-infra12:03
*** tzn has joined #openstack-infra12:03
*** msuriar has joined #openstack-infra12:03
*** edwarnicke has joined #openstack-infra12:03
*** kzaitsev_mb has joined #openstack-infra12:03
*** aysyd has joined #openstack-infra12:03
*** jed56 has joined #openstack-infra12:03
*** sweston has joined #openstack-infra12:03
*** briancurtin has joined #openstack-infra12:03
*** xgerman has joined #openstack-infra12:03
*** mgkwill has joined #openstack-infra12:03
*** sileht has joined #openstack-infra12:03
*** wewe0901 has joined #openstack-infra12:03
*** zhiyan has joined #openstack-infra12:03
*** Ng has joined #openstack-infra12:03
*** mkwiek has joined #openstack-infra12:03
*** skath_ has joined #openstack-infra12:03
*** sballe has joined #openstack-infra12:03
*** monester has joined #openstack-infra12:03
*** igorbelikov has joined #openstack-infra12:03
*** erlon has joined #openstack-infra12:03
*** maximov has joined #openstack-infra12:03
*** auggy has joined #openstack-infra12:03
*** betherly has joined #openstack-infra12:03
*** ameade has joined #openstack-infra12:03
*** ramishra has joined #openstack-infra12:03
*** hrybacki has joined #openstack-infra12:03
*** Adri2000 has joined #openstack-infra12:03
*** elmiko has joined #openstack-infra12:03
*** erikm has joined #openstack-infra12:03
*** mikal has joined #openstack-infra12:03
*** cznewt has joined #openstack-infra12:03
*** sigmavirus24_awa has joined #openstack-infra12:03
*** gus has joined #openstack-infra12:03
*** cloudnull has joined #openstack-infra12:03
*** vkramskikh has joined #openstack-infra12:03
*** mmedvede has joined #openstack-infra12:03
*** bkero has joined #openstack-infra12:03
*** eliqiao_ has joined #openstack-infra12:03
*** sdague has joined #openstack-infra12:03
*** rm_work has joined #openstack-infra12:03
*** kuromagi has joined #openstack-infra12:03
*** clif_h has joined #openstack-infra12:03
*** ryanpetrello has joined #openstack-infra12:03
*** tbarron has joined #openstack-infra12:03
*** comstud has joined #openstack-infra12:03
*** sulo has joined #openstack-infra12:03
*** peterlisak has joined #openstack-infra12:03
*** dtroyer has joined #openstack-infra12:03
*** leifmadsen has joined #openstack-infra12:03
*** Guest40848 has joined #openstack-infra12:03
*** jswarren_ has joined #openstack-infra12:03
*** mattymo has joined #openstack-infra12:03
*** mugsie has joined #openstack-infra12:03
*** d34dh0r53 has joined #openstack-infra12:03
*** vincentfrancoise has joined #openstack-infra12:03
*** sbadia has joined #openstack-infra12:03
*** mrodden has joined #openstack-infra12:03
*** craige has joined #openstack-infra12:03
*** Daviey has joined #openstack-infra12:03
*** lewo has joined #openstack-infra12:03
*** bradm has joined #openstack-infra12:03
*** blogan has joined #openstack-infra12:03
*** whoops has joined #openstack-infra12:03
*** jbryce has joined #openstack-infra12:03
*** beisner has joined #openstack-infra12:03
*** Aegil has joined #openstack-infra12:03
*** smcginnis has joined #openstack-infra12:03
*** mgagne has joined #openstack-infra12:03
*** dimtruck has joined #openstack-infra12:03
*** NikitaKonovalov has joined #openstack-infra12:03
*** Guest5506 has joined #openstack-infra12:03
*** hockeynut has joined #openstack-infra12:03
*** StevenK has joined #openstack-infra12:03
*** hrubi has joined #openstack-infra12:03
*** darrenc has joined #openstack-infra12:03
*** dolphm has joined #openstack-infra12:03
*** jokke_ has joined #openstack-infra12:03
*** wendar has joined #openstack-infra12:03
*** matt6434 has joined #openstack-infra12:03
*** zhenguo_ has joined #openstack-infra12:03
*** DuncanT has joined #openstack-infra12:03
*** rackertom has joined #openstack-infra12:03
*** patrickeast has joined #openstack-infra12:03
*** notnownikki has joined #openstack-infra12:03
*** lane_kong has joined #openstack-infra12:03
*** rossella_s has joined #openstack-infra12:03
*** simonmcc has joined #openstack-infra12:03
*** serverascode has joined #openstack-infra12:03
*** lmiccini has joined #openstack-infra12:03
*** ashtokolov has joined #openstack-infra12:03
*** mescanef has joined #openstack-infra12:03
*** megm has joined #openstack-infra12:03
*** dulek has joined #openstack-infra12:03
*** bmjen has joined #openstack-infra12:03
*** eantyshev has joined #openstack-infra12:03
*** mhayden has joined #openstack-infra12:03
*** jaosorior has joined #openstack-infra12:03
*** phschwartz_ has joined #openstack-infra12:03
*** cfarquhar has joined #openstack-infra12:03
*** pahuang has joined #openstack-infra12:03
*** lindsayk has joined #openstack-infra12:03
*** khappone has joined #openstack-infra12:03
*** pkoniszewski has joined #openstack-infra12:03
*** dstufft has joined #openstack-infra12:03
*** odyssey4me has joined #openstack-infra12:03
*** niska has joined #openstack-infra12:03
*** sgordon has joined #openstack-infra12:03
*** kzaitsev_ws has joined #openstack-infra12:03
*** cp16net has joined #openstack-infra12:03
*** dmellado has joined #openstack-infra12:03
*** calebb has joined #openstack-infra12:03
*** Zara has joined #openstack-infra12:03
*** jmccrory has joined #openstack-infra12:03
*** ianychoi has joined #openstack-infra12:03
*** xek has joined #openstack-infra12:03
*** nibalizer has joined #openstack-infra12:03
*** clarkb has joined #openstack-infra12:03
*** dtantsur has joined #openstack-infra12:03
*** pedroalvarez has joined #openstack-infra12:03
*** jvrbanac has joined #openstack-infra12:03
*** lbragstad has joined #openstack-infra12:03
*** gnuoy has joined #openstack-infra12:03
*** alaski has joined #openstack-infra12:03
*** rfolco has joined #openstack-infra12:03
*** electrofelix has joined #openstack-infra12:03
*** jordan_ has joined #openstack-infra12:03
*** bryan_att has joined #openstack-infra12:03
*** jpr has joined #openstack-infra12:03
*** jlvillal has joined #openstack-infra12:03
*** rguillebert has joined #openstack-infra12:03
*** tpsilva has joined #openstack-infra12:03
*** petems has joined #openstack-infra12:03
*** jraim has joined #openstack-infra12:03
*** ganso has joined #openstack-infra12:03
*** oanson has joined #openstack-infra12:03
*** charz has joined #openstack-infra12:03
*** infra-red has joined #openstack-infra12:03
*** vponomaryov has joined #openstack-infra12:03
*** mestery has joined #openstack-infra12:03
*** teran_ has joined #openstack-infra12:03
*** ams_ has joined #openstack-infra12:03
*** fbo has joined #openstack-infra12:03
*** ajo has joined #openstack-infra12:03
*** markmcclain has joined #openstack-infra12:03
*** Anticimex has joined #openstack-infra12:03
*** tojuvone_ has joined #openstack-infra12:03
*** tdasilva has joined #openstack-infra12:03
*** dougwig has joined #openstack-infra12:03
*** lucasagomes has joined #openstack-infra12:03
*** johnthetubaguy has joined #openstack-infra12:03
*** maishsk has joined #openstack-infra12:03
*** anthonyper has joined #openstack-infra12:03
*** AJaeger has joined #openstack-infra12:03
*** briancline has joined #openstack-infra12:03
*** amrith has joined #openstack-infra12:03
*** jzerebecki has joined #openstack-infra12:03
*** etoews has joined #openstack-infra12:03
*** lindsayk_ has joined #openstack-infra12:03
*** othermordred has joined #openstack-infra12:03
*** warpc has joined #openstack-infra12:03
*** kragniz has joined #openstack-infra12:03
*** smanjara has joined #openstack-infra12:03
*** zz_ja has joined #openstack-infra12:03
*** adreznec has joined #openstack-infra12:03
*** Mmike has joined #openstack-infra12:03
*** tobiash has joined #openstack-infra12:03
*** DinaBelova has joined #openstack-infra12:03
*** hughhalf has joined #openstack-infra12:03
*** bodepd has joined #openstack-infra12:03
*** vipuls has joined #openstack-infra12:03
*** pasquier-s has joined #openstack-infra12:03
*** roaet_ has joined #openstack-infra12:03
*** bookwar has joined #openstack-infra12:03
*** tchaypo has joined #openstack-infra12:03
*** madorn has joined #openstack-infra12:03
*** HeOS_ has joined #openstack-infra12:03
*** lifeless has joined #openstack-infra12:03
*** mknawabi has joined #openstack-infra12:03
*** Odd_Bloke has joined #openstack-infra12:03
*** purp has joined #openstack-infra12:03
*** max_lobur has joined #openstack-infra12:03
*** qb_ has joined #openstack-infra12:03
*** binbincong has joined #openstack-infra12:03
*** sshnaidm has joined #openstack-infra12:03
*** e0ne has joined #openstack-infra12:03
*** dtardivel has joined #openstack-infra12:03
*** matrohon has joined #openstack-infra12:03
*** shardy has joined #openstack-infra12:03
*** tykeal has joined #openstack-infra12:03
*** Nakato has joined #openstack-infra12:03
*** bexelbie has joined #openstack-infra12:03
*** abregman has joined #openstack-infra12:03
*** aeng has joined #openstack-infra12:03
*** yuanying_ has joined #openstack-infra12:03
*** KanagarajM has joined #openstack-infra12:03
*** changbl has joined #openstack-infra12:03
*** ekarlso has joined #openstack-infra12:03
*** hamzy has joined #openstack-infra12:03
*** markusz has joined #openstack-infra12:03
*** aorourke has joined #openstack-infra12:03
*** Jeffrey4l has joined #openstack-infra12:03
*** zxiiro has joined #openstack-infra12:03
*** verdurin has joined #openstack-infra12:03
*** jamespage has joined #openstack-infra12:03
*** clayton has joined #openstack-infra12:03
*** itsuugo has joined #openstack-infra12:03
*** EmilienM has joined #openstack-infra12:03
*** cburgess has joined #openstack-infra12:03
*** iremizov has joined #openstack-infra12:03
*** ruhe has joined #openstack-infra12:03
*** jpmaxman has joined #openstack-infra12:03
*** Ryan_Lane has joined #openstack-infra12:03
*** sylwesterB has joined #openstack-infra12:03
*** dham1 has joined #openstack-infra12:03
*** kgalanov has joined #openstack-infra12:03
*** Vivek_ has joined #openstack-infra12:03
*** mkj_wk has joined #openstack-infra12:03
*** tpeoples has joined #openstack-infra12:03
*** markvan_ has joined #openstack-infra12:03
*** mwhahaha has joined #openstack-infra12:03
*** _ody_ has joined #openstack-infra12:03
*** bdemers has joined #openstack-infra12:03
*** daemontool has joined #openstack-infra12:03
*** onovy has joined #openstack-infra12:03
*** mdenny has joined #openstack-infra12:03
*** flepied has joined #openstack-infra12:03
*** eharney has joined #openstack-infra12:03
*** Madasi has joined #openstack-infra12:03
*** d0ugal has joined #openstack-infra12:03
*** davideagnello has joined #openstack-infra12:03
*** alkhodos has joined #openstack-infra12:03
*** asselin has joined #openstack-infra12:03
*** HeOS has joined #openstack-infra12:03
*** mkoderer__ has joined #openstack-infra12:03
*** coolsvap has joined #openstack-infra12:03
*** MCoLo has joined #openstack-infra12:03
*** ericksonsantos has joined #openstack-infra12:03
*** sambetts has joined #openstack-infra12:03
*** wznoinsk has joined #openstack-infra12:03
*** akscram has joined #openstack-infra12:03
*** loquacities has joined #openstack-infra12:03
*** jbernard has joined #openstack-infra12:03
*** krtaylor has joined #openstack-infra12:03
*** yonglihe has joined #openstack-infra12:03
*** denisra has joined #openstack-infra12:03
*** christiana has joined #openstack-infra12:03
*** laurelm has joined #openstack-infra12:03
*** openstackgerrit_ has joined #openstack-infra12:03
*** openstackgerrit has joined #openstack-infra12:03
*** larainema has joined #openstack-infra12:03
*** dmsimard has joined #openstack-infra12:03
*** Guest93242 has joined #openstack-infra12:03
*** mika has joined #openstack-infra12:03
*** arif-ali has joined #openstack-infra12:03
*** marcusvrn_ has joined #openstack-infra12:03
*** dguitarbite has joined #openstack-infra12:03
*** dgonzalez has joined #openstack-infra12:03
*** masayukig has joined #openstack-infra12:03
*** psanchez has joined #openstack-infra12:03
*** rubasov has joined #openstack-infra12:03
*** SergeyLukjanov has joined #openstack-infra12:03
*** gmann has joined #openstack-infra12:03
*** andreaf has joined #openstack-infra12:03
*** skraynev has joined #openstack-infra12:03
*** cbader has joined #openstack-infra12:03
*** jcooley has joined #openstack-infra12:03
*** otherwiseguy has joined #openstack-infra12:03
*** aspiers has joined #openstack-infra12:03
*** zaro has joined #openstack-infra12:03
*** pfallenop has joined #openstack-infra12:03
*** vmbrasseur has joined #openstack-infra12:03
*** njohnston has joined #openstack-infra12:03
*** jlk has joined #openstack-infra12:03
*** nikhil has joined #openstack-infra12:03
*** zul has joined #openstack-infra12:03
*** hughsaunders has joined #openstack-infra12:03
*** stevemar has joined #openstack-infra12:03
*** mcarden has joined #openstack-infra12:03
*** ayoung has joined #openstack-infra12:03
*** samyewell has joined #openstack-infra12:03
*** sc` has joined #openstack-infra12:03
*** bknudson_ has joined #openstack-infra12:03
*** Apsu has joined #openstack-infra12:03
*** tlbr has joined #openstack-infra12:03
*** kmartin has joined #openstack-infra12:03
*** mkoderer has joined #openstack-infra12:03
*** isq_ has joined #openstack-infra12:03
*** dstanek has joined #openstack-infra12:03
*** TheJulia has joined #openstack-infra12:03
*** pkarikh has joined #openstack-infra12:03
*** davidlenwell has joined #openstack-infra12:03
*** fmccrthy has joined #openstack-infra12:03
*** SergK has joined #openstack-infra12:03
*** russellb has joined #openstack-infra12:03
*** curl_e has joined #openstack-infra12:03
*** tristanC has joined #openstack-infra12:03
*** vaidy_ril has joined #openstack-infra12:03
*** isviridov_away has joined #openstack-infra12:03
*** mnencia has joined #openstack-infra12:03
*** eglute has joined #openstack-infra12:03
*** jroll has joined #openstack-infra12:03
*** evgenyl has joined #openstack-infra12:03
*** mahatic has joined #openstack-infra12:03
*** fungi has joined #openstack-infra12:03
*** akuznetsova has joined #openstack-infra12:03
*** coreycb has joined #openstack-infra12:03
*** jamielennox has joined #openstack-infra12:03
*** fdegir has joined #openstack-infra12:03
*** Kiall has joined #openstack-infra12:03
*** robcresswell has joined #openstack-infra12:03
*** persia has joined #openstack-infra12:03
*** jhesketh has joined #openstack-infra12:03
*** zigo has joined #openstack-infra12:03
*** Clint has joined #openstack-infra12:03
*** adam_g has joined #openstack-infra12:03
*** mhu has joined #openstack-infra12:03
*** ChanServ has joined #openstack-infra12:03
*** asimov.freenode.net sets mode: +o ChanServ12:03
*** asimov.freenode.net sets mode: +bbbb *!*@119.15.127.8 *!~supybot@aluminium.linux2go.dk gasgasgas!*@* *!*@*PanicBNC.org12:03
*** asimov.freenode.net sets mode: +bbbb *!~Energy@* LLKCKfan!*@* *!~rongze@* *!*@213.140.512:03
*** asimov.freenode.net changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/"12:03
yolandagoing to show you a paste12:03
*** piet has joined #openstack-infra12:03
*** bradjones has joined #openstack-infra12:04
*** bradjones has quit IRC12:04
*** bradjones has joined #openstack-infra12:04
mordredyolanda: oh weird12:04
*** zzzeek has joined #openstack-infra12:04
yolandahttp://paste.openstack.org/show/487250/12:04
robcresswello/ For some reason I cant start the Horizon meeting in openstack-meeting-312:04
*** therve has joined #openstack-infra12:04
*** rguillebert has quit IRC12:04
*** teran_ has quit IRC12:04
*** zxiiro has quit IRC12:04
*** mwhahaha has quit IRC12:04
robcresswellAnyone on hand to help out?12:04
*** mtreinish has joined #openstack-infra12:04
*** dougshelley66 has joined #openstack-infra12:04
*** michaelxin has joined #openstack-infra12:04
*** dteselkin has joined #openstack-infra12:04
*** dkaigarodsev has joined #openstack-infra12:04
*** jesusaur has joined #openstack-infra12:04
*** ldnunes has joined #openstack-infra12:04
*** rcarrillocruz has joined #openstack-infra12:04
*** rakhmerov has joined #openstack-infra12:04
*** tsufiev has joined #openstack-infra12:04
*** jogo has joined #openstack-infra12:04
*** kambiz has joined #openstack-infra12:04
*** pabelanger has joined #openstack-infra12:04
*** Sam-I-Am has joined #openstack-infra12:04
*** pc_m has joined #openstack-infra12:04
*** thingee has joined #openstack-infra12:04
*** hogepodge has joined #openstack-infra12:04
mordredyolanda: oh - nodepool is hardcoded to look for a network named 'Ext-Net' or 'public'12:04
*** jesusaur has quit IRC12:04
*** pabelanger has quit IRC12:04
*** pabelanger has joined #openstack-infra12:04
yolandaoh dear12:04
*** pleia2 has joined #openstack-infra12:04
*** smarcet has joined #openstack-infra12:04
*** prometheanfire has joined #openstack-infra12:04
*** flip214 has joined #openstack-infra12:04
*** flip214 has quit IRC12:04
*** flip214 has joined #openstack-infra12:04
*** trey has joined #openstack-infra12:04
*** Liuqing has joined #openstack-infra12:04
*** jogo is now known as Guest2206912:04
*** slagle has joined #openstack-infra12:04
mordredyolanda: so - if the public network in infra-cloud is not named that, we should name it that12:04
*** jesusaur has joined #openstack-infra12:04
*** wfoster has joined #openstack-infra12:05
mordredyolanda: I'm going to eat lunch, then look at the problem in your paste12:05
yolandamordred thanks for the pointer12:05
yolandamm, it's called public12:05
*** tonyb has joined #openstack-infra12:05
*** darvon has joined #openstack-infra12:05
*** placko_ has joined #openstack-infra12:05
*** htruta has joined #openstack-infra12:06
*** coreyob has joined #openstack-infra12:06
*** sdake has joined #openstack-infra12:06
*** afazekas has joined #openstack-infra12:07
*** trey has quit IRC12:07
*** pabelanger has quit IRC12:07
*** Sam-I-Am has quit IRC12:07
*** pc_m has quit IRC12:07
*** thingee has quit IRC12:07
*** hogepodge has quit IRC12:07
*** trey has joined #openstack-infra12:07
*** pc_m has joined #openstack-infra12:07
*** pabelanger has joined #openstack-infra12:07
AJaegerI see a lot of timeouts today, e.g. http://logs.openstack.org/66/281066/2/gate/gate-openstack-manuals-tox-checkniceness/c189c73/console.html#_2016-02-17_11_24_59_47012:07
*** mwhahaha has joined #openstack-infra12:08
AJaeger"ValueError: I/O operation on closed file"12:08
AJaeger"Exception: Error cloning git://git.openstack.org/openstack/openstack-manuals"12:08
AJaegerHas anybody an idea what's going on with git.openstack.org?12:09
*** rguillebert has joined #openstack-infra12:09
*** tzn has quit IRC12:09
yolandaAJaeger, i don't know, but i can check load and logs12:09
*** kushal has joined #openstack-infra12:11
*** coolsvap is now known as coolsvap|away12:11
*** piet has quit IRC12:12
pedroalvarezWe are making progress here in the StoryBoard meetup!  http://i.imgur.com/FUfWS0m.jpg12:12
*** Daisy has quit IRC12:13
AJaegerpedroalvarez: progress would be empty bins ;)12:13
*** piet has joined #openstack-infra12:13
openstackgerritPaul Michali proposed openstack-infra/project-config: Cleanup of constraint based jobs for neutron-vpnaas  https://review.openstack.org/28092212:13
*** GheRivero has joined #openstack-infra12:13
*** Daisy has joined #openstack-infra12:13
pedroalvarezI agree :)12:13
*** zxiiro has joined #openstack-infra12:14
AJaegerpedroalvarez: nice cakes btw ;)12:14
*** wolverineav has joined #openstack-infra12:15
pedroalvarezAJaeger: they are really nice indeed, thanks SotK and Zara12:15
*** teran_ has joined #openstack-infra12:15
*** piet has quit IRC12:15
*** Guest45178 has quit IRC12:17
*** teran_ has quit IRC12:17
mordredyolanda:         if self.hasExtension('os-floating-ips'):12:17
*** teran_ has joined #openstack-infra12:17
*** teran_ is now known as teran12:17
mordredyolanda: I think we maybe have the floating ips extension enabled in the infra cloud12:17
yolandalooks like12:17
*** teran has quit IRC12:17
yolandagood catch12:17
mordredyolanda: let's see about turning it off12:17
*** teran has joined #openstack-infra12:18
yolandait's picking defaults from puppet so we need to set extra config there12:18
*** esikachev has joined #openstack-infra12:18
mordredyolanda: nod12:18
mordredyolanda: you have a puppet change that changes the name of a neutron subnet resource12:19
*** Qiming has quit IRC12:19
yolandamordred, the one to parameterize entries?12:19
mordredyolanda: should I just assume that for now you'll manually delete the old subnet?12:19
mordredyah12:19
*** Qiming has joined #openstack-infra12:19
AJaegerpc_m: I just commented on your neutron-vpnaas review - best check how sdague does it for nova for cover and releasenotes12:19
yolandathat's not for changing the name, is that because we are on different subnets in east/west12:19
yolandaso i need to send different content in east12:19
*** wolverineav has quit IRC12:20
pc_mAJaeger: Which one? I've got several out right now.12:20
yolandai will not change anything on east, and i bet on west they will do the same. I want to do more and more fresh redeploys as soon as puppet changes are merging12:20
mordredyah - but it goes from being neutron_subnet {'provider-subnet-53-54-55' to provider-subnet-infracloud ... does the puppet resource name map into an openstack name?12:20
*** piet has joined #openstack-infra12:20
*** dansmith has joined #openstack-infra12:20
*** dansmith is now known as Guest1430412:20
*** jaypipes has joined #openstack-infra12:21
pc_mAJaeger: 280926.12:21
*** vgridnev has joined #openstack-infra12:21
AJaegerpc_m: https://review.openstack.org/#/c/280926/12:21
pc_mAJaeger: Will look.12:21
AJaegerpc_m: yes - wanted to stress out that not all jobs have constraints enabled. Notable exceptions are releasenotes and cover12:21
AJaegercover is run in post and thus not tested by you - and releasenotes failed ;)12:21
pc_mAJaeger: Does that mean changing project-config to not use the constraints based jobs?12:21
yolandamordred, i came with that name, to go with something generic, so we can reuse for east/west, and that is not tied to subnet pools12:22
AJaegerpc_m: your project-config change looks fine to me.12:22
mordredyolanda: k.12:22
yolandain east right now we only have one pool, and is on the 15.126.48.0/24 range12:22
*** piet has quit IRC12:22
yolandawest seems to be more lucky :)12:22
pc_mAJaeger: thanks. WIll look at Nova.12:22
*** piet has joined #openstack-infra12:22
AJaegerpc_m: you and sdague are pioneering right now the constraint setup without constraints ;)12:22
pc_mAJaeger: Yeah, I'm a sucker for abuse :)12:23
AJaegerpc_m: https://review.openstack.org/26709612:23
mordredyolanda: also - in max-connections patch you said you are seeing errors about too many connections12:23
yolandayes12:23
sdagueAJaeger: it's not done there because the infra doesn't yet support it12:23
mordredyolanda: lowering the max_connections value on the mysql server will cause you to get more of those errors12:23
pc_mAJaeger: Thanks! Saves me some digging!12:23
AJaegerpc_m: you might want to include lifeless and sdague in reviews...12:23
yolandamm right12:23
*** dalgaaf has joined #openstack-infra12:23
mordredyolanda: however, I do believe that lower max connections is a good idea12:23
AJaegersdague: I know12:23
mordredwe might just also need to increase backlog, and connection_timeout12:24
yolandamordred, do you know if there is a way to limit mysql requests from nova, neutron?12:24
mordredthat will cause unused connections to get timed out12:24
pc_mAJaeger: Will do. I plan on doing VPN, FW, LB, and neutron-lib eventually.12:24
yolandafine tuning can be tricky!12:24
mordredand backlog will allow mysql server to stack incoming connections in the tcp backlog without creating server threads for them while it's servicing active requests12:24
*** devananda has quit IRC12:24
*** kzaitsev_mb has quit IRC12:24
*** erlon has quit IRC12:24
*** maximov has quit IRC12:24
*** gus has quit IRC12:24
*** cloudnull has quit IRC12:24
*** kuromagi has quit IRC12:24
*** ryanpetrello has quit IRC12:24
*** clif_h has quit IRC12:24
*** rackertom has quit IRC12:24
*** DuncanT has quit IRC12:24
*** simonmcc has quit IRC12:24
*** pahuang has quit IRC12:24
*** khappone has quit IRC12:24
*** odyssey4me has quit IRC12:24
*** sgordon has quit IRC12:24
*** rfolco has quit IRC12:24
*** jpr has quit IRC12:24
*** petems has quit IRC12:24
*** vponomaryov has quit IRC12:24
*** ajo has quit IRC12:24
*** Anticimex has quit IRC12:24
*** ams_ has quit IRC12:24
*** anthonyper has quit IRC12:24
*** jzerebecki has quit IRC12:24
*** othermordred has quit IRC12:24
*** warpc has quit IRC12:24
*** kragniz has quit IRC12:24
*** adreznec has quit IRC12:24
*** hughhalf has quit IRC12:24
*** DinaBelova has quit IRC12:24
*** bodepd has quit IRC12:24
*** vipuls has quit IRC12:24
*** roaet_ has quit IRC12:24
*** Nakato has quit IRC12:24
*** KanagarajM has quit IRC12:24
*** markusz has quit IRC12:24
*** aorourke has quit IRC12:24
*** jamespage has quit IRC12:24
*** itsuugo has quit IRC12:24
*** iremizov has quit IRC12:24
*** jpmaxman has quit IRC12:24
*** Ryan_Lane has quit IRC12:24
*** kgalanov has quit IRC12:24
*** Vivek_ has quit IRC12:24
*** markvan_ has quit IRC12:24
*** bdemers has quit IRC12:24
*** daemontool has quit IRC12:24
*** d0ugal has quit IRC12:24
*** asselin has quit IRC12:24
*** HeOS has quit IRC12:24
*** sambetts has quit IRC12:24
*** yonglihe has quit IRC12:24
*** larainema has quit IRC12:24
*** marcusvrn_ has quit IRC12:24
*** dguitarbite has quit IRC12:24
*** dgonzalez has quit IRC12:24
*** andreaf has quit IRC12:24
*** jcooley has quit IRC12:24
*** aspiers has quit IRC12:24
*** zaro has quit IRC12:24
*** pfallenop has quit IRC12:24
*** vmbrasseur has quit IRC12:24
*** jlk has quit IRC12:24
*** zul has quit IRC12:24
*** stevemar has quit IRC12:24
*** mcarden has quit IRC12:24
*** samyewell has quit IRC12:24
*** Apsu has quit IRC12:24
*** sc` has quit IRC12:24
*** njohnston has quit IRC12:24
*** fmccrthy has quit IRC12:24
*** SergK has quit IRC12:24
*** russellb has quit IRC12:24
*** tristanC has quit IRC12:24
*** vaidy_ril has quit IRC12:24
*** isviridov_away has quit IRC12:24
*** jroll has quit IRC12:24
*** evgenyl has quit IRC12:24
*** fungi has quit IRC12:24
*** akuznetsova has quit IRC12:24
*** coreycb has quit IRC12:24
*** jamielennox has quit IRC12:24
*** fdegir has quit IRC12:24
*** Kiall has quit IRC12:24
*** jhesketh has quit IRC12:24
*** zigo has quit IRC12:24
*** Clint has quit IRC12:24
*** adam_g has quit IRC12:24
*** mhu has quit IRC12:24
*** Clint has joined #openstack-infra12:24
*** Clint has quit IRC12:25
*** Clint has joined #openstack-infra12:25
*** kzaitsev_mb has joined #openstack-infra12:25
*** Apsu has joined #openstack-infra12:25
*** sc` has joined #openstack-infra12:25
*** jamespage has joined #openstack-infra12:25
*** odyssey4me has joined #openstack-infra12:25
*** dgonzalez has joined #openstack-infra12:25
*** jzerebecki has joined #openstack-infra12:25
*** pfallenop has joined #openstack-infra12:25
*** anthonyper has joined #openstack-infra12:25
*** jlk has joined #openstack-infra12:25
*** roaet_ has joined #openstack-infra12:25
*** KanagarajM has joined #openstack-infra12:25
*** Anticimex has joined #openstack-infra12:25
*** akuznetsova has joined #openstack-infra12:25
*** cloudnull has joined #openstack-infra12:25
*** daemontool has joined #openstack-infra12:25
*** markvan has joined #openstack-infra12:25
*** gus has joined #openstack-infra12:25
*** tristanC has joined #openstack-infra12:25
*** sgordon has joined #openstack-infra12:25
*** jroll has joined #openstack-infra12:25
*** aorourke has joined #openstack-infra12:25
*** asselin has joined #openstack-infra12:25
*** andreaf has joined #openstack-infra12:25
*** jlk has quit IRC12:25
*** jlk has joined #openstack-infra12:25
mordredyolanda: I do not yet know how to tune the db pool settings in nova/neutron12:25
*** mhu has joined #openstack-infra12:25
*** Nakato has joined #openstack-infra12:25
*** Kiall has joined #openstack-infra12:25
*** clif_h has joined #openstack-infra12:25
*** Kiall has quit IRC12:25
*** Kiall has joined #openstack-infra12:25
*** HeOS has joined #openstack-infra12:25
*** jamespage has quit IRC12:25
*** jamespage has joined #openstack-infra12:25
*** rfolco has joined #openstack-infra12:25
*** pahuang has joined #openstack-infra12:25
*** adam_g has joined #openstack-infra12:25
*** ryanpetrello has joined #openstack-infra12:25
*** njohnston has joined #openstack-infra12:25
AJaegerpc_m: neutron itself as well?12:25
*** kragniz has joined #openstack-infra12:25
*** bodepd has joined #openstack-infra12:25
*** fungi has joined #openstack-infra12:25
*** dguitarbite has joined #openstack-infra12:25
*** larainema has joined #openstack-infra12:25
*** bdemers has joined #openstack-infra12:25
*** vponomaryov has joined #openstack-infra12:25
*** DinaBelova has joined #openstack-infra12:25
*** markvan has quit IRC12:25
*** markvan has joined #openstack-infra12:25
*** SergK has joined #openstack-infra12:25
*** stevemar has joined #openstack-infra12:25
*** devananda has joined #openstack-infra12:25
*** pahuang has quit IRC12:25
*** pahuang has joined #openstack-infra12:25
*** d0ugal has joined #openstack-infra12:25
*** adam_g has quit IRC12:25
*** adam_g has joined #openstack-infra12:25
*** vmbrasseur has joined #openstack-infra12:25
*** ajo has joined #openstack-infra12:25
yolandamordred i can update the commit message in the change for database12:25
*** jcooley has joined #openstack-infra12:25
*** larainema has quit IRC12:25
*** larainema has joined #openstack-infra12:25
*** vmbrasseur has quit IRC12:25
*** vmbrasseur has joined #openstack-infra12:25
*** dguitarbite has quit IRC12:25
*** dguitarbite has joined #openstack-infra12:25
*** russellb has joined #openstack-infra12:25
*** sambetts has joined #openstack-infra12:25
*** ajo has quit IRC12:25
*** ajo has joined #openstack-infra12:25
mordredyolanda: this might be one of those situations where we want to turn nodepool on, then manually muck around with some different settings choices on mysql until we find some that we like12:25
yolandai'm looking now at how to disable the floating ip extension12:25
*** aspiers has joined #openstack-infra12:26
*** itsuugo has joined #openstack-infra12:26
*** kuromagi has joined #openstack-infra12:26
*** zigo has joined #openstack-infra12:26
*** khappone has joined #openstack-infra12:26
mordredcool12:26
*** hughhalf has joined #openstack-infra12:26
*** zaro has joined #openstack-infra12:26
yolandaah yes, i have setup a nodepool server on rax for testing12:26
*** mcarden has joined #openstack-infra12:26
*** jhesketh has joined #openstack-infra12:26
*** coreycb has joined #openstack-infra12:26
*** markusz has joined #openstack-infra12:26
mordredneat12:26
*** piet has quit IRC12:26
mordredmaybe a little later I can work with you on poking at some of those mysql settings12:26
yolandait will be nice12:26
*** zul has joined #openstack-infra12:26
yolandafor the amount of computes we have, we need to start looking at HA12:26
mordredyah12:27
*** yamamoto has joined #openstack-infra12:27
*** vaidy_ril has joined #openstack-infra12:27
yolanda90 computes with a single controller, that has rabbit, mysql, in it... doesn't sound good12:27
*** vipuls has joined #openstack-infra12:27
*** isviridov_away has joined #openstack-infra12:27
mordredheh12:27
mordredwell - I think we decided to not do HA laregly because if the controller goes down for a few minutes, it's not that big of a deal for nodepool12:27
mordredwe're HA because we have multiple clouds12:27
yolandayes but with the amount of computes we have, we need to redistribute the load, or we won't be able to use the max power of it12:28
*** yamamoto_ has joined #openstack-infra12:29
*** taron2 has quit IRC12:29
*** jamielennox has joined #openstack-infra12:30
*** yamamoto has quit IRC12:31
*** jpr has joined #openstack-infra12:31
*** briancli1e has joined #openstack-infra12:32
mordredah - that's a little different of a concern12:32
*** _bmjen has joined #openstack-infra12:32
*** lewo_ has joined #openstack-infra12:32
*** jbernard_ has joined #openstack-infra12:32
*** dolphm_ has joined #openstack-infra12:32
*** peterlisak_ has joined #openstack-infra12:32
*** dstufft_ has joined #openstack-infra12:32
*** adreznec has joined #openstack-infra12:32
*** Daisy has quit IRC12:32
*** sirushti has quit IRC12:32
*** asilenkov_ has quit IRC12:32
*** wewe0901 has quit IRC12:32
*** igorbelikov has quit IRC12:32
*** elmiko has quit IRC12:32
*** vkramskikh has quit IRC12:32
*** sdague has quit IRC12:32
*** peterlisak has quit IRC12:32
*** vincentfrancoise has quit IRC12:32
*** smcginnis has quit IRC12:32
*** jbryce has quit IRC12:32
*** dimtruck has quit IRC12:32
*** NikitaKonovalov has quit IRC12:32
*** hockeynut has quit IRC12:32
*** Guest5506 has quit IRC12:32
*** dolphm has quit IRC12:32
*** lewo has quit IRC12:32
*** mgagne has quit IRC12:32
*** patrickeast has quit IRC12:32
*** bmjen has quit IRC12:32
*** pkoniszewski has quit IRC12:32
*** dstufft has quit IRC12:32
*** clarkb has quit IRC12:32
*** bryan_att has quit IRC12:32
*** jlvillal has quit IRC12:32
*** jraim has quit IRC12:32
*** ganso has quit IRC12:32
*** oanson has quit IRC12:32
*** fbo has quit IRC12:32
*** AJaeger has quit IRC12:32
*** briancline has quit IRC12:32
*** etoews has quit IRC12:32
*** lindsayk_ has quit IRC12:32
*** tchaypo has quit IRC12:32
*** madorn has quit IRC12:32
*** Odd_Bloke has quit IRC12:32
*** mknawabi has quit IRC12:32
*** qb_ has quit IRC12:32
*** e0ne has quit IRC12:32
*** dtardivel has quit IRC12:32
*** tykeal has quit IRC12:32
*** hamzy has quit IRC12:32
*** Jeffrey4l has quit IRC12:32
*** verdurin has quit IRC12:32
*** EmilienM has quit IRC12:32
*** dham1 has quit IRC12:32
*** tpeoples has quit IRC12:32
*** onovy has quit IRC12:32
*** flepied has quit IRC12:32
*** jbernard has quit IRC12:32
*** krtaylor has quit IRC12:32
*** openstackgerrit_ has quit IRC12:32
*** openstackgerrit has quit IRC12:32
*** dmsimard has quit IRC12:32
*** mika has quit IRC12:32
*** rubasov has quit IRC12:32
*** SergeyLukjanov has quit IRC12:32
*** cbader has quit IRC12:32
*** otherwiseguy has quit IRC12:32
*** nikhil has quit IRC12:32
*** hughsaunders has quit IRC12:32
*** ayoung has quit IRC12:32
*** tlbr has quit IRC12:32
*** pkarikh has quit IRC12:32
*** Madasi has quit IRC12:32
*** davidlenwell has quit IRC12:32
*** mnencia has quit IRC12:32
*** eglute has quit IRC12:32
*** curl_e has quit IRC12:32
*** mahatic has quit IRC12:32
*** arif-ali has quit IRC12:32
*** eglute_s has joined #openstack-infra12:32
*** peterlisak_ is now known as peterlisak12:32
*** mika has joined #openstack-infra12:32
*** mika has joined #openstack-infra12:32
*** zz_dimtruck has joined #openstack-infra12:32
*** madorn_ has joined #openstack-infra12:32
*** mgagne_ has joined #openstack-infra12:32
*** mgagne_ has quit IRC12:32
*** mgagne_ has joined #openstack-infra12:32
*** Vivek has joined #openstack-infra12:32
*** pkarikh_ has joined #openstack-infra12:32
*** AJaeger has joined #openstack-infra12:32
*** mnencia_ has joined #openstack-infra12:32
*** lewo_ is now known as lewo12:32
*** dolphm_ is now known as dolphm12:32
*** jbryce_ has joined #openstack-infra12:32
*** jbryce_ is now known as jbryce12:32
*** krtaylor_ has joined #openstack-infra12:32
*** ayoung has joined #openstack-infra12:32
*** zz_dimtruck is now known as dimtruck12:32
*** curl_e has joined #openstack-infra12:32
*** smcginnis has joined #openstack-infra12:32
*** Odd_Bloke has joined #openstack-infra12:32
*** Vivek is now known as Guest8602012:32
*** oanson has joined #openstack-infra12:32
*** onovy has joined #openstack-infra12:32
*** nikhil has joined #openstack-infra12:32
*** NikitaKonovalov has joined #openstack-infra12:32
*** elmiko has joined #openstack-infra12:32
*** elmiko has quit IRC12:32
*** elmiko has joined #openstack-infra12:32
*** dmsimard has joined #openstack-infra12:32
*** mnencia_ is now known as mnencia12:32
*** Daisy has joined #openstack-infra12:32
*** tchaypo has joined #openstack-infra12:32
*** e0ne has joined #openstack-infra12:32
*** tykeal has joined #openstack-infra12:32
*** dstufft_ is now known as dstufft12:32
*** verdurin has joined #openstack-infra12:33
*** fbo_ has joined #openstack-infra12:33
*** tlbr_ has joined #openstack-infra12:33
*** davidlenwell has joined #openstack-infra12:33
*** mknawabi has joined #openstack-infra12:33
*** hughsaunders has joined #openstack-infra12:33
*** clarkb has joined #openstack-infra12:33
*** med_ has joined #openstack-infra12:33
mordredthat's about scaling rather than HA - I think SpamapS actually has some tests he's been doing where he's been able to get to a 1000 nodes on a single mysql/rabbit controller - but I'd need to check with him on numbers12:33
*** rubasov has joined #openstack-infra12:33
*** cbader has joined #openstack-infra12:33
*** yamamoto_ has quit IRC12:33
*** vincentfrancoise has joined #openstack-infra12:33
*** openstackgerrit has joined #openstack-infra12:33
*** sdague has joined #openstack-infra12:33
mordredyolanda: I'm pretty sure we can get the mysql and rabbit to handle the load for 90 ... although I'm often wrong :)12:33
*** med_ is now known as Guest3757012:33
*** openstackgerrit_ has joined #openstack-infra12:33
*** pkoniszewski has joined #openstack-infra12:33
yolandawell the server is quite powerful12:34
*** jlvillal has joined #openstack-infra12:34
*** vkramskikh has joined #openstack-infra12:34
*** hamzy has joined #openstack-infra12:35
mordredyah12:35
*** arif-ali has joined #openstack-infra12:35
mordredit's not small :)12:35
mordredalso- HA mode of rabbit actually reduces performance12:35
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Remove python-constraints-jobs from ofagent job definition  https://review.openstack.org/28124012:35
*** flepied has joined #openstack-infra12:35
rcarrillocruzthat's disturbing...so rabbit is not ready for HA?12:36
rcarrillocruzfor prod workloads, etc12:36
*** otherwiseguy has joined #openstack-infra12:36
yolandait should be, i did the setup for ha in rabbit years ago...12:36
*** hockeynut has joined #openstack-infra12:36
yolandafor openstack deployments12:36
*** EmilienM has joined #openstack-infra12:36
*** sirushti has joined #openstack-infra12:36
*** vgridnev_ has joined #openstack-infra12:36
*** SergeyLukjanov has joined #openstack-infra12:37
*** asilenkov_ has joined #openstack-infra12:37
*** Madasi has joined #openstack-infra12:37
*** mahatic has joined #openstack-infra12:37
mordredit's ready - it works - it's just slower12:38
mordredlike, for clouds that aren't infra cloud it's probably worth the tradeoff12:38
mordredbut for us, since our concern is getting as much performance as we can out of our cloud and are willing to sacrifice a little uptime for that12:38
mordredI think it's not worth the tradeoff12:38
*** vgridnev has quit IRC12:38
*** fdegir has joined #openstack-infra12:39
*** evgenyl has joined #openstack-infra12:40
*** othermordred has joined #openstack-infra12:40
AJaegerpc_m, sdague: See also 28124012:40
*** ociuhandu has joined #openstack-infra12:40
yolandawell we will see the results based on usage, if it's worth to add HA, or isolate mysql or rabbit in some unique server.. whatever works better12:40
*** fmccrthy has joined #openstack-infra12:41
*** DuncanT has joined #openstack-infra12:41
mordredyolanda: totally agree12:41
AJaegeryolanda: and in Barcelano you'll give a presentation about what you learned for a high-performance testing cloud ;)12:41
*** erlon has joined #openstack-infra12:42
AJaegerSo, please take notes what worked - and what failed ;)12:42
yolandashall i learn something? :)12:42
*** gordc has joined #openstack-infra12:42
*** rhallisey has joined #openstack-infra12:42
mordredyolanda: also, I've been thinking more and more about managing our own HA mysql cluster for infra things - so if we do that then we'll have the puppet/ansible for it12:42
AJaegeryolanda: if not - you can still teach us ;)12:42
yolandamordred really i'm glad that this is becoming a reality. With the amount of hard we have, we can spin up a huge cloud12:43
*** kgalanov has joined #openstack-infra12:43
yolandaah, an infra mysql cluster, that sounds good12:43
*** weshay has joined #openstack-infra12:43
*** Ryan_Lane has joined #openstack-infra12:44
*** rackertom has joined #openstack-infra12:45
*** simonmcc has joined #openstack-infra12:45
*** samyewell has joined #openstack-infra12:45
*** etoews has joined #openstack-infra12:46
*** iremizov has joined #openstack-infra12:46
*** KanagarajM has quit IRC12:46
*** jpmaxman has joined #openstack-infra12:47
*** yonglihe has joined #openstack-infra12:47
*** wewe0901 has joined #openstack-infra12:48
*** EricGonczer_ has joined #openstack-infra12:49
*** lindsayk_ has joined #openstack-infra12:49
*** marcusvrn_ has joined #openstack-infra12:50
*** jordan_ has quit IRC12:50
*** jordan_ has joined #openstack-infra12:50
*** yamamoto has joined #openstack-infra12:50
*** baoli has joined #openstack-infra12:50
pc_mAJaeger: speaking of cover job... do any projects use that as one of the jobs for a check queue?12:50
*** petems has joined #openstack-infra12:50
*** tpeoples has joined #openstack-infra12:51
*** thorst has joined #openstack-infra12:51
*** baoli has quit IRC12:51
*** maximov_ has joined #openstack-infra12:51
*** jraim has joined #openstack-infra12:51
*** ganso has joined #openstack-infra12:51
AJaegerpc_m: yes, a few do12:52
AJaegerpc_m: our problem right now is that constrainsts cannot be used in *post* queue - and both releasenotes and cover is in post used as well12:52
*** sweston has quit IRC12:53
*** edwarnicke has quit IRC12:53
*** mgkwill has quit IRC12:53
*** dougwig has quit IRC12:53
*** hrybacki has quit IRC12:53
*** iremizov has quit IRC12:53
*** samyewell has quit IRC12:53
pc_mAJaeger: Would like to enable that for networking-cisco repo, say in the check queue. Would that be OK?12:53
* pc_m this is unrelated to constraints...12:53
pc_mAJaeger: goal would be to have coverage data for reviewers to examine, to help decide if testing is adequate.12:55
*** daemontool has quit IRC12:55
* pc_m we've got only 60% coverage in that repo.12:55
mordredyolanda: yah - gerrit slowdowns at 00:00 UTC are apparently due to our mysql backups - it would be nice to do backups from a replication slave instead12:55
*** dtardivel has joined #openstack-infra12:56
*** igorbelikov has joined #openstack-infra12:56
*** taron2 has joined #openstack-infra12:56
*** dham1 has joined #openstack-infra12:58
*** Daisy has quit IRC12:58
*** ildikov has joined #openstack-infra12:58
pc_mAJaeger: Does the job just generate a report? Is there any mechanism for comparing coverage to some baseline (previous run/saved baseline/etc)?12:58
*** piet has joined #openstack-infra12:59
AJaegerpc_m: job generates a report but I think some teams have set it up to compare against baseline - but I don't have details.13:00
* AJaeger expects that most repos have coverage post jobs and nobody is aware of them ;(13:01
pc_mAJaeger: Cool. Any idea who I could ask?13:01
*** rlandy has joined #openstack-infra13:01
*** claudiub has joined #openstack-infra13:01
pc_mAJaeger: I see VPN has it in the post queue and it runs the cover target.  The target is defined in networking-cisco.13:01
*** kushal has quit IRC13:02
openstackgerritDmitry Tantsur proposed openstack-infra/project-config: Switch most of ironic jobs to iPXE and remove a separate iPXE job  https://review.openstack.org/28125313:02
*** Guest86020 is now known as Vivek13:03
pc_mAJaeger: If I want to add this to networking-cisco, should I add it to the experimental queue, and then later add to check queue (as non-voting?)?13:03
*** Vivek has quit IRC13:03
*** Vivek has joined #openstack-infra13:03
*** Guest10770 is now known as zeus`13:04
*** zeus` is now known as zeus13:04
*** zeus has quit IRC13:04
*** zeus has joined #openstack-infra13:04
*** piet has quit IRC13:05
*** piet has joined #openstack-infra13:06
*** lucasagomes is now known as lucas-hungry13:06
*** kzaitsev_mb has quit IRC13:07
*** piet has quit IRC13:08
*** sylwesterB has quit IRC13:08
*** daemontool has joined #openstack-infra13:09
*** alkhodos has quit IRC13:10
*** amrith is now known as _amrith_13:12
*** ildikov has quit IRC13:13
AJaegerpc_m: I hope others can help with coverage...13:13
*** Sam-I-Am has joined #openstack-infra13:14
pc_mAJaeger: Yeah, looking for recommendations on how to roll it out (experimental -> check, or just check), and whether there is a way to compare to baseline.13:15
*** dprince has joined #openstack-infra13:15
*** andymaier has joined #openstack-infra13:15
*** baoli has joined #openstack-infra13:16
*** wolverineav has joined #openstack-infra13:16
*** fawadkhaliq has quit IRC13:17
*** abregman is now known as abregman|mtg13:18
*** Kiall has quit IRC13:18
*** Kiall has joined #openstack-infra13:19
yolandamordred, about disabling an extension, do you know best way? the only thing that worked for me is commenting policy.json, on policy.json:    "os_compute_api:os-floating-ips": "",13:19
*** yamamoto has quit IRC13:19
yolandai wonder if there is some rule to disable it, even if the line is present13:20
*** kgalanov has quit IRC13:20
*** dims_ has joined #openstack-infra13:20
openstackgerritBogdan Dobrelya proposed openstack-infra/project-config: Add docs-on-rtfd hook for fuel-noop-fixtures project  https://review.openstack.org/28126613:21
*** wolverineav has quit IRC13:21
*** achanda has joined #openstack-infra13:21
*** dims has quit IRC13:21
mordredyolanda: I do not know enough to be useful13:21
*** kgalanov has joined #openstack-infra13:22
yolandai'll ask in openstack-dev13:23
*** dkranz has joined #openstack-infra13:23
*** flwang has joined #openstack-infra13:23
*** _oanson has joined #openstack-infra13:24
*** hrubi has quit IRC13:24
*** yamamoto has joined #openstack-infra13:25
*** esikachev has quit IRC13:25
*** daemontool_ has joined #openstack-infra13:25
*** hrubi has joined #openstack-infra13:26
*** achanda has quit IRC13:26
*** ildikov has joined #openstack-infra13:27
*** changzhi has joined #openstack-infra13:27
*** oanson has quit IRC13:27
openstackgerritSergey Kraynev proposed openstack-infra/project-config: Revert "Add heat lbaas specific test for v1 and v2"  https://review.openstack.org/28127013:28
*** daemontool has quit IRC13:29
*** markvoelker has joined #openstack-infra13:29
*** zeus has quit IRC13:29
*** bswartz has joined #openstack-infra13:29
*** kzaitsev_mb has joined #openstack-infra13:29
*** dims_ has quit IRC13:29
therveyolanda, AJaeger ^^^ The patch above fixes the heat gate13:30
*** vgridnev_ has quit IRC13:32
*** vgridnev has joined #openstack-infra13:32
*** alivigni has joined #openstack-infra13:34
*** esikachev has joined #openstack-infra13:34
AJaegertherve: approved13:34
therveThanks!13:35
*** changzhi has quit IRC13:35
*** kushal has joined #openstack-infra13:35
*** xyang1 has joined #openstack-infra13:35
*** __oanson has joined #openstack-infra13:36
*** sfinucan has quit IRC13:36
*** sfinucan has joined #openstack-infra13:37
*** zeus has joined #openstack-infra13:37
*** zeus is now known as Guest4972813:37
openstackgerritClint Adams proposed openstack-infra/project-config: Set ZUUL_REF for devstack plugins list proposal periodic job  https://review.openstack.org/28127513:38
*** Guest49728 is now known as zeus13:38
*** zeus has quit IRC13:38
*** zeus has joined #openstack-infra13:38
AJaegertherve: once it merged wait until zuul has been reconfigured (see status.openstack.org/zuul for reconfigured line) before you recheck - or wait an hour....13:38
*** yamamoto has quit IRC13:38
therveOK good to know13:39
*** oanson has joined #openstack-infra13:39
*** vgridnev_ has joined #openstack-infra13:39
*** _oanson has quit IRC13:40
*** __oanson has quit IRC13:41
*** fawadkhaliq has joined #openstack-infra13:42
*** abregman|mtg is now known as abregman13:42
*** vgridnev has quit IRC13:42
*** yamamoto has joined #openstack-infra13:43
*** doug-fish has quit IRC13:44
*** hashar has joined #openstack-infra13:44
*** dhellmann has quit IRC13:44
*** dims has joined #openstack-infra13:45
*** dhellmann has joined #openstack-infra13:47
*** tykeal is now known as tykeal_away13:47
openstackgerritBogdan Dobrelya proposed openstack-infra/project-config: Add docs-on-rtfd hook for fuel-noop-fixtures project  https://review.openstack.org/28126613:47
*** tykeal_away is now known as tykeal13:47
*** jlibosva has joined #openstack-infra13:48
*** HeOS has quit IRC13:50
*** HeOS has joined #openstack-infra13:50
*** yamamoto has quit IRC13:50
*** matrohon has quit IRC13:50
openstackgerritBogdan Dobrelya proposed openstack-infra/project-config: Add docs-on-rtfd hook for fuel-noop-fixtures project  https://review.openstack.org/28126613:50
*** smarcet has quit IRC13:52
*** jordan_ has quit IRC13:52
*** jordan_ has joined #openstack-infra13:53
*** Kiall has quit IRC13:54
*** smanjara has quit IRC13:55
*** nelsnelson has joined #openstack-infra13:55
*** annegentle has joined #openstack-infra13:55
*** nelsnelson has quit IRC13:56
*** baoli_ has joined #openstack-infra13:56
*** yamamoto has joined #openstack-infra13:57
*** nelsnelson has joined #openstack-infra13:57
*** kgiusti has joined #openstack-infra13:58
*** vgridnev_ has quit IRC13:58
*** _tzn has joined #openstack-infra13:59
*** Kiall has joined #openstack-infra13:59
*** nelsnelson has quit IRC13:59
*** derekh has joined #openstack-infra13:59
*** baoli has quit IRC13:59
*** matrohon has joined #openstack-infra14:00
*** thingee has joined #openstack-infra14:00
*** hogepodge has joined #openstack-infra14:00
openstackgerritMerged openstack-infra/project-config: Revert "Add heat lbaas specific test for v1 and v2"  https://review.openstack.org/28127014:01
*** bryan_att has joined #openstack-infra14:02
*** amitgandhinz has joined #openstack-infra14:02
*** vincentfrancoise has left #openstack-infra14:03
*** jordan_ has quit IRC14:04
*** electrical has joined #openstack-infra14:04
*** tiswanso has joined #openstack-infra14:04
*** edmondsw has joined #openstack-infra14:04
*** KanagarajM has joined #openstack-infra14:05
*** edwarnicke has joined #openstack-infra14:05
*** annegentle has quit IRC14:07
*** vgridnev_ has joined #openstack-infra14:07
*** annegentle has joined #openstack-infra14:08
*** samyewell has joined #openstack-infra14:08
*** fdegir has quit IRC14:09
*** dims has quit IRC14:09
*** dims has joined #openstack-infra14:10
*** julim has quit IRC14:10
*** nelsnelson has joined #openstack-infra14:11
*** warpc has joined #openstack-infra14:11
*** EricGonczer_ has quit IRC14:11
*** fdegir has joined #openstack-infra14:11
*** dougwig has joined #openstack-infra14:12
*** ayoung has quit IRC14:12
*** nelsnelson has quit IRC14:12
*** sweston has joined #openstack-infra14:13
*** phschwartz_ is now known as phschwartz14:14
*** infra-re_ has joined #openstack-infra14:14
DuncanTHi. I've got a couple of tempest scenarios for cinder (main one does plenty of exercise of the quota code) that I don't want to run on the main gate. I plan on creating a gate job with a the fake backend driver eventually to allow this to be run in gate, but for now I just want to get it up and reviewed. Is there an established way of putting in non-gate14:15
DuncanTtests? (an example one is fine)14:15
*** lucas-hungry is now known as lucasagomes14:16
SamYaplefungi: did you already send out the last batch of ATC stuff for Austin? Or is there another run?14:16
*** jordanP has joined #openstack-infra14:17
*** rkukura has joined #openstack-infra14:17
*** infra-red has quit IRC14:17
*** wolverineav has joined #openstack-infra14:17
*** ams_ has joined #openstack-infra14:17
KanagarajMAJaeger, hi, when you find time, could you please review https://review.openstack.org/28066214:17
*** _tzn has quit IRC14:18
AJaegerDuncanT: I'm a bit confused - best practice is to add a job as experimental first so that you can "check experimental", and then add as non-voting for check queue. Is that what you're looking for?14:19
*** dtardivel has quit IRC14:20
*** dtardivel has joined #openstack-infra14:20
*** jaosorior has quit IRC14:20
*** mgkwill has joined #openstack-infra14:20
openstackgerritMax Lobur proposed openstack-infra/project-config: Add bareon-ironic project  https://review.openstack.org/28130114:20
*** rkukura has quit IRC14:20
DuncanTAJaeger: No, this is way too slow to be run on the main gate14:20
DuncanTAJaeger: It will eventually be run on a 'cinder-big-ops' special gate job14:21
*** jaosorior has joined #openstack-infra14:21
DuncanTAJaeger: But step one of that is to get a test into tempest14:21
DuncanTAJaeger: As far as I can tell14:21
*** wolverineav has quit IRC14:21
DuncanTAJaeger: How to put something in tempest that doesn't get run on the main devstack gate is my question14:22
*** hrybacki has joined #openstack-infra14:22
*** jsavak has joined #openstack-infra14:23
DuncanTAJaeger: Nova has a big-ops gate test, but it isn't clear to me how the big-ops tests are differentiated from normal tests14:24
*** Liuqing has quit IRC14:24
*** Guest14304 is now known as dansmith14:24
*** dansmith is now known as Guest8441114:25
openstackgerritMax Lobur proposed openstack-infra/project-config: Add bareon-ironic project  https://review.openstack.org/28130114:26
*** dimtruck is now known as zz_dimtruck14:26
*** Guest84411 is now known as dansmith14:27
*** doug-fish has joined #openstack-infra14:28
*** mriedem has joined #openstack-infra14:28
*** asettle has quit IRC14:29
*** mescanef has quit IRC14:29
*** vgridnev__ has joined #openstack-infra14:30
*** auggy has quit IRC14:31
*** auggy has joined #openstack-infra14:32
*** zeih has joined #openstack-infra14:33
*** patrickeast_ has joined #openstack-infra14:33
*** dougwig has quit IRC14:33
*** patrickeast_ has quit IRC14:33
*** samyewell has quit IRC14:33
*** warpc has quit IRC14:33
*** hrybacki has quit IRC14:33
*** ams_ has quit IRC14:33
*** mgkwill has quit IRC14:33
*** electrical has quit IRC14:33
*** sabeen has joined #openstack-infra14:33
*** vgridnev_ has quit IRC14:33
*** zeih has quit IRC14:34
*** yamamoto has quit IRC14:35
*** jordan_ has joined #openstack-infra14:35
*** dims_ has joined #openstack-infra14:35
*** dims has quit IRC14:36
*** iremizov has joined #openstack-infra14:36
fungiSamYaple: i sent a batch on sunday but there will be another in early march, after feature freeze14:36
*** _amrith_ is now known as amrith14:37
*** sabeen3 has joined #openstack-infra14:37
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Only try to delete floating ip if the server has it  https://review.openstack.org/28131014:37
*** zeih has joined #openstack-infra14:37
SamYaplefungi: awesome14:37
*** sabeen has quit IRC14:38
*** jordan_ has quit IRC14:39
*** jordan_ has joined #openstack-infra14:39
*** jordanP has quit IRC14:39
fungijeblair: so that cinder blip in iad was one of the planned maintenances this week rackspace had warned us about. when we discussed a few weeks ago we decided to just see how afs deals with it... apparently not well but easy enough to fix?14:40
fungijeblair: worth noting, that was not the last of the impacting maintenances they had warned about for this week14:40
fungiso it may recur in the next day or so14:40
*** abregman has quit IRC14:40
openstackgerritMonty Taylor proposed openstack-infra/nodepool: Only try to delete floating ip if the server has it  https://review.openstack.org/28131014:40
AJaegerDuncanT: best work with the QA folks in that case, sorry, can't help further here14:41
AJaegerDuncanT: #openstack-qa14:41
DuncanTAJaeger: Ah ha, I didn't know that channel existed. Thanks.14:41
*** ildikov has quit IRC14:42
AJaegerfungi: those timeouts we had this morning - could those be a result of the same maintenance ?14:43
*** rbrndt has joined #openstack-infra14:43
*** Qiming has quit IRC14:43
cody-somervillesdague: woot. Thanks for the +2 +A! :)14:44
*** Qiming has joined #openstack-infra14:44
sdaguecody-somerville: no prob, thanks for the docs14:45
*** Swami has joined #openstack-infra14:45
fungiAJaeger: which timeouts? i only sort of skimmed scrollback14:45
AJaegerI said: I see a lot of timeouts today, e.g. http://logs.openstack.org/66/281066/2/gate/gate-openstack-manuals-tox-checkniceness/c189c73/console.html#_2016-02-17_11_24_59_47014:46
AJaegerThose were timeouts by zuul-cloner - and quite a few14:46
fungiAJaeger: quike possibly. see the cloud networks maintenances scheduled for this week at https://status.rackspace.com/14:47
fungier, quite14:47
AJaegerfungi: looks fine now - so let's hope that was it...14:47
fungiAJaeger: the cinder impact was in fact because of network maintenance, so i wouldn't be surprised if some of our systems were also just flat out unreachable in or from rackspace14:48
AJaegerfungi: yolanda tried to fix https://github.com/openstack/giftwrap/ - it#14:48
AJaeger.. it's empty and our jobs don't push there14:48
AJaegeropenstack is setup14:48
yolandaah thanks for the reminder AJaeger. fungi, so push fails there, i wonder how can this be fixed14:48
fungialso, we should expect more before the week is out14:48
fungiAJaeger: yolanda: the fix is just to add the gerrit collaboration group under settings for that repo14:48
yolandaoh14:49
AJaegerfungi: yolanda was not able to access it and needed some knowledge sharing on how to do it properly14:49
yolandaso that is a manual step?14:49
*** lindsayk_ has quit IRC14:49
*** othermordred has quit IRC14:49
*** monester has quit IRC14:49
yolandaalso the instructions on system-config were a bit buggy. I could test ssh finally from review to github, but with git user, that's not documented properly14:50
fungiyolanda: what tends to happen is that the github api call to create the repo succeeds but then the subsequent call to add gerrit as a collaborator fails (github is lousy with intermittent api errors) and jeepyb doesn't know to try again so it has to be done manually14:50
*** monester has joined #openstack-infra14:50
*** jordan_ has quit IRC14:50
yolandanew thing learnt again...14:50
AJaegerfungi: yes, that rackspace maintenance could indeed be the root cause14:50
*** othermordred has joined #openstack-infra14:50
fungii have no doubt the documentation we have around github integration is lacking. many of us wish we could just get rid of it entirely14:50
*** lindsayk_ has joined #openstack-infra14:51
*** vgridnev__ has quit IRC14:51
*** sridhar_ram has joined #openstack-infra14:52
*** vgridnev__ has joined #openstack-infra14:52
*** KanagarajM has quit IRC14:52
*** KanagarajM has joined #openstack-infra14:53
pabelangermorning14:53
*** flaper87 has quit IRC14:54
*** flaper87 has joined #openstack-infra14:54
AJaegerfungi, could you put "Fix tag publishing of server projects" - https://review.openstack.org/279238 - on your review queue, please? That'S the discussion from last week about neutron publishing from tags - and the logic in that change needs carefull review. So, drink your coffee first, please ;)14:54
AJaegermorning, pabelanger14:54
fungiAJaeger: ahh, right, i read it and was mulling it over, then some things caught on fire and now i don't even recall what. thanks for the reminder!14:55
*** julim has joined #openstack-infra14:55
AJaegerfungi: too many firedrills ;(14:55
funginot drills unfortunately--actual fires14:55
fungiand we're the brigade14:55
*** vgridnev__ has quit IRC14:55
fungianyway, 279238 is pulled up next in my queue so i don't forget again14:56
*** vgridnev__ has joined #openstack-infra14:56
yolandafungi, i created https://review.openstack.org/281125 this morning, to fix the user to connect to github in documentation. What do you think if i extend it, and i reflect that bugfix for github?14:57
*** vgridnev__ has quit IRC14:58
*** Guest22953 is now known as mfisch14:58
*** mfisch is now known as Guest3294014:59
fungiyolanda: well, those instructions were not in fact incorrect, though perhaps misleading14:59
*** placko_ has quit IRC14:59
*** mriedem is now known as mriedem_meeting14:59
yolandamissing the git user needed to connect14:59
fungiyou can ssh github.com to accomplish the task set forth in that section (accepting and caching github.com's ssh server key as a known_hosts entry)14:59
fungidon't need git@ for that14:59
fungihowever, i'm not opposed to updating it to be less misleading. better still would be to add a section on testing authenticated git+ssh access to github (which does indeed need the git@ or -l git)15:00
*** ddieterly has joined #openstack-infra15:01
*** piet has joined #openstack-infra15:01
yolandaok15:01
*** EricGonczer_ has joined #openstack-infra15:02
fungii was merely pointing out that the example you're modifying there is not actually for testing access to github, it's merely for getting a copy of the ssh server key from github15:02
jrollclarkb: jeblair: hey fyi, room 7777 on the pbx system isn't passing audio through, 7778 is working fine for us, just wanted to give you a heads up15:02
jrollnot urgent ofc15:02
fungijroll: for sip clients, or pots/pstn or both?15:02
fungipabelanger: ^ heads up15:03
*** dims_ has quit IRC15:03
jrollfungi: seems like both15:03
yolandamm, i understand now. It was confusing to me because what i wanted is to test the ssh github connection15:03
fungiyolanda: completely understand, and i think it would be great to add a section on doing that (also the modification you've proposed doesn't hurt, and reduces confusion in my opinion)15:03
*** pradk has joined #openstack-infra15:04
yolandayep, and completing that with testing ssh access + troubleshoot on replication is needed15:04
yolandai'll try to find some time for it15:04
*** annegentle has quit IRC15:04
*** dhellmann has quit IRC15:05
*** annegentle has joined #openstack-infra15:05
*** dhellmann has joined #openstack-infra15:05
fungiyolanda: thanks! that would be awesome15:06
*** ccrouch has joined #openstack-infra15:06
*** annegentle has quit IRC15:07
pabelangerfungi: jroll: Hmm, can't think of a reason my 1 room would not work15:07
clarkbjroll I wonder if someone muted the channel? is that even a thing random users can do?15:07
clarkbpabelanger: ^15:07
*** Swami_ has joined #openstack-infra15:07
jrollclarkb: not that I can see15:07
pabelangerI don't think we enable admin access to conferences via dialplan15:07
pabelangerI can confirm15:07
jrollpabelanger: yeah, strange, but like I said it isn't urgent15:08
*** vgridnev__ has joined #openstack-infra15:08
pabelangerokay, in room 7777 now15:08
pabelanger1 sec15:08
jrollpabelanger: okay, I need to go run this midcycle but I can answer during lulls :)15:09
*** dims has joined #openstack-infra15:09
fungijroll: thanks for the heads up, apologies for the complication, and get back to your event--best of luck!15:10
jrollthanks fungi :)15:11
jrolljust a hiccup, otherwise it's been fantastic15:11
*** Swami has quit IRC15:11
*** sigmavirus24_awa is now known as sigmavirus2415:11
fungijroll: this may be the first non-test-related heavy use we've put it under, so i appreciate your team's willingness to take these bumps in stride15:12
jrollnot a problem at all :)15:14
jeblairpabelanger: i joined 777715:14
*** daemontool_ has quit IRC15:16
*** piet has quit IRC15:16
*** yamahata has joined #openstack-infra15:17
*** jtomasek_ has joined #openstack-infra15:17
*** wolverineav has joined #openstack-infra15:18
*** jlanoux_ has joined #openstack-infra15:18
*** piet has joined #openstack-infra15:18
*** jlanoux has quit IRC15:18
*** jordanP has joined #openstack-infra15:18
NobodyCamis there a way to mute all?15:18
jeblairpabelanger: oh, yeah, i see what you mean. fail2ban is probably a good idea.15:19
NobodyCamon the conf bridge?15:19
pabelangeryes, but you need admin access15:19
NobodyCamahh ok15:19
pabelangerunless silence detection is out of wack for some reason15:20
pabelangerwhich is enabled on bridges15:20
*** othermordred has quit IRC15:21
pabelangerjeblair: ya, will work on that today15:21
JayFfungi: we recently had a problem with a user putting out conference on hold and piping loud hold music into everyones' ears. We moved into another conference for now, but is there any way to ... like kick those people? or at least ID them?15:21
*** othermordred has joined #openstack-infra15:22
*** wolverineav has quit IRC15:22
jeblairJayF: oh, that might have been me, sorry :(15:22
*** zz_dimtruck is now known as dimtruck15:22
JayFjeblair: whoops. Hahahaha. Nice elevator music :P15:23
jeblairi'm gone now15:23
JayFIt's fine, we moved to 777915:23
*** jordanP has quit IRC15:23
pabelangerYa, we need to enable some admin logic to allow a chair to do that15:24
jeblairpabelanger: maybe we should run "iptables -I openstack-INPUT 7 -s 74.208.201.251 -j REJECT" ?15:24
pabelangerjeblair: sure, that works for me15:24
*** achanda has joined #openstack-infra15:24
*** kozhukal` has joined #openstack-infra15:24
* JayF <3 when people properly use iptables insert syntax with line numbers15:24
*** MarkAtwood has joined #openstack-infra15:24
pabelangerI mean, we can kick everybody from 7777 and see what happens15:24
*** hrybacki has joined #openstack-infra15:24
pabelangersince you are no longer using the room15:24
*** kozhukal` is now known as kozhukalov15:25
AJaegerfungi: yeah, you're right about drills and actual fires...15:25
*** mescanef has joined #openstack-infra15:26
*** mtanino has joined #openstack-infra15:26
*** ajmiller has joined #openstack-infra15:26
*** zeih has quit IRC15:27
JayFWe have reports from some users of the conf that the DID number is going to busy, before asking for a conference room or anything. I haven't tested myself as I'm on the sip.15:28
jeblairi'll see if we've run out of money again15:28
*** vgridnev__ has quit IRC15:28
pabelangerjeblair: did you want to run iptables?15:28
jeblairpabelanger: just did; looks like there's another ip too15:29
jeblairpabelanger: though i had to move it to position 2; i think it was getting through on the conntrack rule15:29
*** achanda has quit IRC15:29
jeblairpabelanger: there, that looks like both ips15:29
jeblairJayF, pabelanger, clarkb, fungi: yep, looks like we're incurring more of a toll charge than i was anticipating15:30
*** jpr has quit IRC15:30
*** vgridnev__ has joined #openstack-infra15:31
JayFjeblair: Does that mean we should discourage use of the DID? Will it never start working again (ever = today/tomorrow)?15:31
pabelangerjeblair: JayF if you are no longer using 7777, I am going to lock and kick the room.15:31
JayFpabelanger: we're only using 7779 right now15:31
pabelangersee if that resets it to the default state15:31
pabelangerjeblair: JayF okay, if somebody wants to join 777715:34
pabelangerto do a quick test15:34
* JayF cannot, is participating in the mid-cycle in 777915:34
jeblairJayF: i don't think we should change things right now.  i think $25/day is something the community can afford for a midcycle.  :)15:34
* jeblair joins 777715:35
JayFjeblair: so it's fixed, then?15:35
JayFbtw, the sound quality on this has been amazing15:35
*** zeih has joined #openstack-infra15:35
*** sdake has quit IRC15:35
JayFjroll had birds outside his window yesterday and twice I checked to see if the chirping was coming out my window b/c of how clear the sound was15:35
*** vgridnev__ has quit IRC15:35
pabelangerJayF: jeblair Yup, 7777 is working again15:36
pabelangerbut PSTN is down, jeblair is paying again15:36
*** jordanP has joined #openstack-infra15:37
*** bexelbie has quit IRC15:37
*** sdake has joined #openstack-infra15:37
*** vgridnev__ has joined #openstack-infra15:37
*** daemontool has joined #openstack-infra15:38
pabelangerjeblair: info on how to kick users from a conference: https://wiki.asterisk.org/wiki/display/AST/ConfBridge#ConfBridge-ConfBridgeCLIOptions15:38
*** bexelbie has joined #openstack-infra15:39
*** Qiming has quit IRC15:41
*** vgridnev__ has quit IRC15:42
*** pabelanger has quit IRC15:42
*** pabelanger has joined #openstack-infra15:42
jeblairJayF: phone bill is paid!  pstn should work again15:42
*** vgridnev__ has joined #openstack-infra15:42
*** jtomasek_ has quit IRC15:43
*** Sukhdev has joined #openstack-infra15:43
*** tzn has joined #openstack-infra15:44
*** _oanson has joined #openstack-infra15:44
*** Guest12383 is now known as BobBall15:45
*** agentle has joined #openstack-infra15:45
*** oanson has quit IRC15:48
jeblairlooks like there are about 5 pstn users and 22 sip users15:48
*** tzn has quit IRC15:48
*** esikachev has quit IRC15:48
*** piet has quit IRC15:52
*** vgridnev__ has quit IRC15:52
*** vgridnev__ has joined #openstack-infra15:52
*** piet has joined #openstack-infra15:52
*** vgridnev__ has quit IRC15:52
tykealI've got a nodepool that I'm bringing up with the following config extract: https://paste.fedoraproject.org/324113/55724301/  can someone give me an idea as to why my multi-node lab is only bringing up the first of the 4 nodes and connecting it up to Jenkins as if it was completely online?15:53
tykealI don't even see the nodepool service trying to bring up 3 more nodes with that definition15:54
*** dizquierdo has joined #openstack-infra15:55
*** mriedem_meeting is now known as mriedem15:56
jeblairtykeal: nodepool's multi-node operation will only attach one node to jenkins15:57
tykealjeblair: that's fine, I expect that. But the 3 extra nodes aren't even started!15:57
jeblairah ok15:57
jeblairtykeal: nodepool calls those 'subnodes' (with various forms of capitalization) in the debug log; does a 'grep -i subnode' there provide any clues?15:59
jeblairtykeal: ah!16:00
clarkbI think the issue is the key in yaml is subnodes not sub-nodes16:00
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/28135416:00
jeblairclarkb: yes that's it16:00
jeblairtykeal: it's 'subnodes'16:01
*** patrickeast has joined #openstack-infra16:01
tykealdoh, I must have misread that16:01
tykeallet me give it a try, and thank you!16:01
*** patrickeast_ has joined #openstack-infra16:01
jeblairdo we not validate config on startup?16:01
mordredjeblair: I'm going to go with "no"16:02
jeblairmordred: good call16:02
* mordred is sooper smaart16:02
*** patrickeast has quit IRC16:02
Clintbot unable to change topic in #openstack-meeting16:02
*** jsavak has quit IRC16:02
tykealthere's some amount of validation as I had a lot of issues getting to the point where I'm at now16:02
*** maishsk has quit IRC16:02
jeblairtykeal: apparently 'nodepool config-validate' will validate the config file16:03
*** jsavak has joined #openstack-infra16:03
*** salv-orl_ has joined #openstack-infra16:03
*** claudiub has quit IRC16:03
*** ociuhandu has quit IRC16:03
tykealhuzzah! thank you clarkb and jeblair that was my mistake16:04
*** baoli_ has quit IRC16:05
*** salv-orlando has quit IRC16:05
*** piet has quit IRC16:06
*** daemontool has quit IRC16:06
fungiinteresting we don't run nodepool config-validate on start, but maybe that's to account for the possibility of phasing in settings in the config before we upgrade to a version supporting them?16:07
*** daemontool has joined #openstack-infra16:07
*** EricGonczer_ has quit IRC16:08
*** KanagarajM has quit IRC16:08
fungii wonder if that should be a default behavior though, and then grow something like a --no-validate option to nodepoold for that use case16:08
fungijeblair: finally got around to trying out last-seen in gertty. working great! database migration was so quick i didn't even see it16:09
*** ildikov has joined #openstack-infra16:09
fungithis is crazy useful16:09
*** _nadya_ has quit IRC16:10
*** ociuhandu has joined #openstack-infra16:11
*** sridhar_ram has quit IRC16:11
*** piet has joined #openstack-infra16:12
*** Jeffrey4l has joined #openstack-infra16:12
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Implemented display param on authentication OIDC request  https://review.openstack.org/28136116:12
*** daemontool_ has joined #openstack-infra16:12
*** Guest40431 is now known as jgriffith16:12
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Adding integration test support for app catalog horizon plugin  https://review.openstack.org/27644016:13
*** jgriffith is now known as Guest281516:13
*** agentle has quit IRC16:13
*** daemontool has quit IRC16:14
*** agentle has joined #openstack-infra16:14
jeblairfungi: woo!  yeah, i'm going to use it right now to look at that change dims told me to look at yesterday ;)16:15
fungiindeed16:15
*** Guest2815 is now known as jgriffith16:15
*** sylwesterB has joined #openstack-infra16:15
*** infra-re_ has quit IRC16:16
dimsjeblair : yay! thanks16:17
*** piet has quit IRC16:17
fungialso conflicts-with is working well16:17
*** esikachev has joined #openstack-infra16:17
*** piet has joined #openstack-infra16:17
*** maishsk has joined #openstack-infra16:17
jeblairclarkb: can you look at my comment in 279299 ?16:18
*** dougwig has joined #openstack-infra16:18
*** wolverineav has joined #openstack-infra16:19
clarkbjeblair aiui with the 6 month retention and expiration of old data mtreinish is ready to get all the job's subunit info16:19
jeblairclarkb: sorry, i was getting at: only grenade jobs should have logs/old/subunit16:19
*** armax has joined #openstack-infra16:19
jeblairclarkb: so is it useful to keep the filter to avoid a few unecessary GETs?16:20
clarkboh that, hrm ya that probably makes sense16:20
jeblairclarkb: or just don't worry about it?16:20
*** mgkwill has joined #openstack-infra16:20
*** ams_ has joined #openstack-infra16:20
*** fedexo has joined #openstack-infra16:20
*** samyewell has joined #openstack-infra16:20
*** warpc has joined #openstack-infra16:20
mtreinishclarkb: that's what I'm hoping, I don't have any first hand experience with all the data though, so it'll kinda be a trial by fire16:20
clarkbit is extra GETs but thosr are relatively cheap, but it is a simple thing to avoid them so probably good to do so16:20
*** sambetts has left #openstack-infra16:20
mtreinishbut my gut is it should be fine :)16:21
jeblairmtreinish: i approved 279299 and its child; but it sounds like if you wanted to add the grenade job name filter back in, it might be useful16:21
fungiigorbelikov: 275329 just reminded me that we'll want to make sure http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/data/bindep-fallback.txt is also sane for debian. did you have a change proposed to do that yet? i assume it's already getting exercised by the cache-bindep element while you're locally testing the diskimage definition in 26472616:21
*** electrical has joined #openstack-infra16:21
clarkbmtreinish: and remove the retry true lines16:21
jeblairclarkb: that's done16:21
jeblairclarkb: (that's the child change)16:22
jeblairhttps://review.openstack.org/28092416:22
clarkboh perfect16:22
*** zeih has quit IRC16:22
jeblairya, i approved them together on purpose :)16:22
*** tzn has joined #openstack-infra16:22
*** Guest46279 is now known as redrobot16:22
mtreinishjeblair: I can push a 3rd change to do that16:22
*** wolverineav has quit IRC16:23
*** Swami has joined #openstack-infra16:23
jeblairmtreinish: cool thanks!16:23
*** Swami_ has quit IRC16:24
*** daemontool__ has joined #openstack-infra16:24
Sam-I-Amanteaya: ping16:24
*** esikachev has quit IRC16:24
*** asettle has joined #openstack-infra16:24
fungijeblair: weird behavior... i was reviewing a project-config change and it showed me verify options too. but then after i left a code-review vote and went back into review for the same change again it only had code-review and workflow options16:25
igorbelikovfungi: cache-bindep element seems to work without any problems on debian, but I'll take a closer look on that, thanks for pointing it out16:25
jeblairfungi: it was probably synced while you were in bootstrappers, then synced again after you were removed16:25
fungijeblair: also nothing in my .gertty.log to indicate it had trouble checking labels for that, so not much else to go on16:26
openstackgerritDmitriy Ukhlov proposed openstack-infra/project-config: Fixes neutron template  for oslo.messaging drivers test  https://review.openstack.org/28137516:26
openstackgerritMerged openstack-infra/system-config: Get debian kernel headers based on architecture  https://review.openstack.org/27532316:26
fungijeblair: hrm, well this was a fresh gertty session, but yeah that could be it since i guess the cache persists across restarts16:26
*** piet has quit IRC16:26
*** daemontool_ has quit IRC16:26
Sam-I-Amjeblair fungi can i bother either one of you for a gate job review?16:27
openstackgerritMerged openstack-infra/system-config: Remove job filters and add missing paths to subunit collection  https://review.openstack.org/27929916:27
jeblairfungi: yes -- available code review labels are all stored with the change.  usually ones permissions don't change very quickly, and it works well enough.  it's not great for when we add ourselves to bootstrappers though, as anything synced during that time will have extra labels, at least until those changes are synced again.16:28
*** daemontool__ has quit IRC16:28
openstackgerritMerged openstack-infra/system-config: Remove retry-get param from subunit file collection  https://review.openstack.org/28092416:28
*** maishsk_ has joined #openstack-infra16:28
*** maishsk has quit IRC16:28
jeblairfungi: if i'm particularly lucid at the time, sometimes i'll stop gertty if i'm elevating permissions for an extended period.16:29
fungiigorbelikov: awesome--let me know if you need help. i expect it's "working" on debian only because the platform tags are over-specific on some entries and so nothing is installing for those (for example liberasurecode-dev says platform:ubuntu-precise platform:ubuntu-trusty rather than just platform:dpkg)16:29
fungiigorbelikov: or in the ruby case, we have ruby installing only on platform:ubuntu-trusty and rubygems only on platform:rpm and platform:ubuntu-precise16:29
fungijeblair: a grand idea. now i just need to hope for periods of lucidity myself16:29
jeblairfungi: yeah, doesn't happen often16:29
anteayaSam-I-Am: at storyboard meetup can it wait until next week16:30
anteayaor can someone else help you?16:30
Sam-I-Amanteaya: just looking for a +2 on something16:30
Sam-I-Ambut i suppose it can wait a bit longer16:30
*** maishsk_ is now known as maishsk16:30
anteayaSam-I-Am: understood, someone else will be faster16:31
* Sam-I-Am looks around16:31
*** yamahata has quit IRC16:32
*** jsavak has quit IRC16:32
igorbelikovfungi: yeah, trusty/precise tags gave me the same idea. Shouldn't be much different from trusty though, so I'll see what exactly won't work with platform:dpkg and will go from there16:32
*** jsavak has joined #openstack-infra16:33
*** Guest32940 is now known as mfisch16:34
*** shashank_hegde has joined #openstack-infra16:34
*** mfisch has quit IRC16:34
*** mfisch has joined #openstack-infra16:34
fungiigorbelikov: right, could just add platform:debian anywhere you see platform:ubuntu-trusty and chances are it will mostly work (except maybe in situations where a package is only in ubuntu and not imported from debian or was packaged independently and has differing names)16:34
*** fedexo has quit IRC16:35
*** pcaruana has quit IRC16:35
kozhukalovguys, could you please review this https://review.openstack.org/#/c/279074/ ?16:35
mtreinishclarkb, jeblair: the other thing is I think the subunit gearman worker needs a restart, there are some changes which landed a couple months ago which I don't think are running yet16:37
*** rm_work has quit IRC16:37
*** rm_work has joined #openstack-infra16:37
mtreinishspecifically this one: http://git.openstack.org/cgit/openstack-infra/puppet-subunit2sql/commit/?id=5327cc64a87f63958fd24929bb5d2765c61f752e16:38
*** bpokorny has joined #openstack-infra16:39
*** jswarren_ has quit IRC16:40
pabelangerjeblair: regarding the zuul memory leak, are you thinking of let zuul run for another week or so before merging the open reviews / cutting a new release?16:40
openstackgerritMatthew Treinish proposed openstack-infra/system-config: Add a job filter for old side subunit files  https://review.openstack.org/28138316:41
jeblairpabelanger: i think jhesketh just posted a fix to another relevant bug; so i'd like to get that in and observe it for a few weeks16:41
*** abregman has joined #openstack-infra16:42
*** bpokorny has quit IRC16:42
jeblairi think we're still leaking somewhat16:42
jeblairhttp://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=392&rra_id=2&view_type=&graph_start=1455122484&graph_end=1455727284  looks leaky16:42
*** kzaitsev_mb has quit IRC16:42
*** bpokorny has joined #openstack-infra16:43
openstackgerritMark Vanderwiel proposed openstack-infra/project-config: Add heat lbaas specific test for v1 and v2  https://review.openstack.org/28135416:43
pabelangerjeblair: Ya, I thought there was some mention last week about still leaking, but couldn't get confirmation. Either way, letting it run a few weeks is what I expected.16:44
*** shashank_hegde has quit IRC16:44
SpamapSyolanda: mordred is right. You can get to at least 1000 computes on one controller. Especially if you don't use neutron's ml2 agents. ;)16:44
*** mgagne_ is now known as mgagne16:45
yolandaSpamapS good to know, i thought that the relation from controller and computes had to be smaller16:45
mordredSpamapS: we have one single provider network16:45
SpamapSyolanda: 90 computes, even with enough active users to overwhelm the 90 computes, will likely work fine with a controller on 4vcpu and 16GB.16:45
*** salv-orl_ has quit IRC16:46
SpamapSmordred: that actually doesn't matter for the ncomputes scaling16:46
mordredSpamapS: ah - good to know16:46
SpamapSmordred: each compute has an ml2 agent, and they _POUND_ on the MQ and DB16:46
yolandaSpamapS, i started to see some mysql errors, nova and glance api failing because of that, so we may need to tune it16:46
mordredSpamapS: even if doing nothing?16:46
mordredSOOOOOO16:46
SpamapSmordred: they're as chatty as nova-compute when doing nothing16:46
AJaegerfungi: thanks for the review. I make the tag/tags consistent and amend the change later16:46
*** jswarren has joined #openstack-infra16:46
mordredyolanda, crinkle, nibalizer: don't kill me here ...16:47
SpamapSyolanda: should need minimal tuning.16:47
AJaegerSam-I-Am: what needs reviewing?16:47
SpamapSyolanda: mysql especially, gets about 1/20th the load as RabbitMQ in my tests.16:47
mordredyolanda, crinkle, nibalizer: when we're in FtC (maybe when I show up so I can join) we should at least have a conversation about what moving to OVN for neutron plugin would look like16:47
mordredsince it makes the ml2 agent go away16:48
SpamapSmordred: wait, wait, killing you is an option?16:48
mordredSpamapS: not for you16:48
mordredjust htem16:48
SpamapSmordred: OH ok, cause I was gonna say..16:48
SpamapSlike I don't WANT to or anything16:48
SpamapS;)16:48
Sam-I-AmAJaeger: you already +2'd it - https://review.openstack.org/#/c/278101/16:48
* SpamapS would like to make it clear, nobody should kill mordred, or anybody else. :)16:48
SpamapSOVN is pretty awesome16:49
yolandamordred i'm fine with that, if the performance is going to be better . I don't have much expertise on cloud deployments, and specially on neutron, so i'm open to advices16:49
pleia2good morning16:49
AJaegerSam-I-Am: the change needs neutron liason +1 first16:49
SpamapSI'm still a little worried about HA for ovsdb in OVN land.16:49
Sam-I-Ammordred: is there an issue with the ml2 plug-in?16:49
AJaegerSam-I-Am: ask dougwig, armax, or mestery to review, 278101 please16:49
mordredSam-I-Am: yes. it does not perform well at scale16:49
SpamapSSam-I-Am: it scales poorly with many computes.16:49
Sam-I-Ammordred: even without l3?16:49
mordredSam-I-Am: it had the benefit until recently of being the best open source option16:50
mordredSam-I-Am: yes16:50
*** salv-orlando has joined #openstack-infra16:50
AJaegerSam-I-Am: ah, it's by mestery - so it's naturally fine ;) Sorry for the noise..16:50
Sam-I-AmAJaeger: lol16:50
mordredSam-I-Am: but now that ovn exists and works, ml2 has much less positive reason for existence anymore16:50
dougwigAJaeger: looking16:50
SpamapSSam-I-Am: there's still an agent that has to plug ports.16:50
mordredSam-I-Am: I do not think it's time to suggest switching upstream default16:50
*** sheeprine has quit IRC16:50
yolandaSpamapS, mordred, so i'm working on a testenv with nodepool, jenkins... so we can stress the infra cloud a bit, and see what needs tuning16:50
mordredSam-I-Am: because ovn is still missing some features16:51
Sam-I-Ammordred: well, some important bits of ovn are missing yet... at least if we want to keep the 'legacy' floating ip feature.16:51
mordredyup16:51
SpamapSSam-I-Am: and doing that over oslo.messaging doesn't make sense at much higher than 100 computes.16:51
Sam-I-Amif i was me, i'd ditch fips :)16:51
mordredSam-I-Am: once those features are there, I believe it will be time to start the conversation16:51
mordredSam-I-Am: me too16:51
Sam-I-Amthey're a hack, and dont work for v616:51
SpamapSyolanda: if you haven't snuggled up to Rally yet, you should. It makes such things really pleasant.16:51
mordredSam-I-Am: but ovn will have them in the next 2 months16:51
Sam-I-Amneutron routers are just nat devices :/16:51
mordredSpamapS: well, for yolanda's case, we do not need to learn what rally is telling us necessarily16:51
AJaegerdougwig: thanks - but I didn' realize this is by mestery16:51
*** SotK_ has quit IRC16:51
mordredSpamapS: we need to know specifically how the one workload of nodepool works16:51
yolandawhat kills me is the amount of ips we have for vms, over 55 only . So we cannot grow as much as we can16:52
*** Zara__ has quit IRC16:52
*** sheeprine has joined #openstack-infra16:52
mordredyolanda: ++16:52
SpamapSmordred: actually the most useful thing Rally tells me is what time the real (not setup) testing started and stopped.16:52
*** maishsk has quit IRC16:52
SpamapSmordred: I use that to query influxdb. :)16:52
Sam-I-Ammordred: while ovn performs better than anything else, it is rather complex for new users learning neutron... in which case there might be a case for ml2 + linux bridge agent for learning purposes.16:52
clarkbyolanda: over 55 only? not sure I parsed that16:52
mordredSam-I-Am: totally16:52
*** rockyg has joined #openstack-infra16:52
SpamapStelegraf+influxdb is really, really nice for metering controllers btw.16:52
mordredSam-I-Am: I do not think we should delete it16:52
armaxmordred: if I read you correctly, contemplating to switch ovn for ml2 would be like switching apples to oranges16:53
mordredSpamapS: for us, nodepool is instrumented with statsd, so we can get the comparable timings16:53
yolandaclarkb, that's a limit on our network in east, i hope that this will be solved when blades are moved16:53
mordredarmax: yes, I know16:53
armaxAJaeger: that change lgtm16:53
Sam-I-Ammordred: good news is we have this networking guide... and i wrote a spec to put ovn into it, but ovn isnt really 'core' yet... so its waiting patiently. meanwhile, i'm working on a reference arch and deployment scenario in the ovn repo.16:53
SpamapSmordred: nodepool is too polite. ;)16:53
mordredSpamapS: :)16:53
*** Guest93242 has quit IRC16:53
SpamapSmordred: but for their cloud, yeah, that's really all that matters. :)16:53
Sam-I-Ammordred: and we'll move that stuff to the networking guide (to replace conventional ovs) when its ready16:53
jeblairyolanda: yeah, i hope it will be fixed when they move, 55 is not worth us doing the project at all.  :(16:53
yolandayep16:53
yolandawe asked a different range16:54
Sam-I-Ammordred: the key to ovn is making sure it has sufficient docs when it becomes a 'thing'16:54
clarkbcould we he tunnel and just v6?16:54
mordredarmax: what I'm talking about is figuring out how to tell people who want to do pure open source for the cloud AND scale that htey should start off thinking abut ovn16:54
SpamapSclarkb: GMTA16:54
Sam-I-Amso it doesnt become another neutron magic black box16:54
SpamapSespecially for nodepool, v6 makes a ton of sense.16:54
mordredarmax: if the open source part is not essential and they're using a vendor network solution, then the answer is potentially different16:54
SpamapSbut I would guess v6 transit is as hard as getting a CIDR in their range.16:54
*** EricGonczer_ has joined #openstack-infra16:55
clarkbSpamapS: ya it is a thibg we tell other clouds when the problem of IPs comes up16:55
greghaynesJust set up a HE v6 tunnel16:55
greghaynes:)16:55
Sam-I-Ammordred: not sure if you read the ovs dev list, but i posted that we should focus on 'real' L3 support instead of trying to reproduce DVR... then try to deprecate the floating ip concept16:55
*** boris-42 has joined #openstack-infra16:55
armaxmordred: one can still do full open source with ml2, it’s a matter of how you assemble the building blocks to build cloud services16:55
mordredarmax: sure. if you want a small cloud16:55
armaxas for the scale part, I am hopeful ovn delivers on its promises16:55
*** tphummel has joined #openstack-infra16:55
Sam-I-Amgreghaynes: i've been known to drag he tunnels into rax cloud boxen for openstack testing16:55
yolandaclarkb, greghaynes, if that can be an option to bypass that ip limit, i'd like to hear more. I don't know about that "HE v6 tunnel" topic sorry16:55
Sam-I-Amwhat we need is clouds that provide native v616:56
armaxbut I’d rather get some serious mileage from ovn before making such claims16:56
SpamapSgreghaynes: yeah good point, we can tunnel and be done.16:56
AJaegeryolanda: if you still have time, could you try making Sam-I-Am happy with reviewing 278101, please?16:56
greghaynesyolanda: thats just about it - you terminate the tunnel on one of your nodes and add a neutron network that routes via it16:56
mordredarmax: I agree16:56
clarkbyolanda: using one ipv4 address you tunnel ipv6 to hurricane electric16:56
*** sfinucan has quit IRC16:56
mordredarmax: so far in our internal testing its kicking the pants off of ml2 by a very large margin16:56
Sam-I-Amassign a /56 to each tenant... go!16:56
mordredarmax: but I agree that real data is needed16:56
*** gyee has joined #openstack-infra16:56
yolandait may be worth discussing or trying on the midcycle16:57
mordredarmax: that's why I was saying that once it's feature complete we sohuld _start_ talking about whether it's a good default16:57
greghaynesSam-I-Am: Not even assign - neutron works *great* with provider networks and SLAAC16:57
armaxmordred: default for what?16:57
Sam-I-Amgreghaynes: well 'assign'16:57
*** baoli has joined #openstack-infra16:57
greghayneshah, yep16:57
Sam-I-Amgreghaynes: in other words, get a /48 from he, not a /6416:57
Sam-I-Ambecause trying to split a /64 is Bad and Wrong16:57
*** roxanaghe has joined #openstack-infra16:57
crinkleyes i would like to get some usage out of infracloud and figure out where bottlenecks are before trying to optimize16:57
armaxmordred: I am not sure the conditions under which the two platforms developed justify a switch, ever16:57
greghaynesSam-I-Am: Yep. That is exactly how my little personal cloud works because I am cheap16:57
fungiSam-I-Am: /48 per tenant is actually perfectly sane16:57
yolandaAJaeger, looking16:57
Sam-I-Amfungi: or that. i can only get a /48 from he, so i do /56s per tenant.16:58
*** julim has quit IRC16:58
Sam-I-Amin a prod env, it'd be different16:58
jeblairclarkb, greghaynes, SpamapS, yolanda: the ipv6 he idea is cool, but let's also not bottleneck on it if we're going to get a larger range when we move.16:58
Sam-I-Ami hear some cloud providers are trying to do longer than /64s for tenants16:58
fungiSam-I-Am: oh, well yeah of you're using he and aren't your own service provider, sure16:58
Sam-I-Ami'm like... ???16:58
AJaegerdougwig: two more for your review queue, please - not urgent: 280922 and 28124016:58
*** agentle has quit IRC16:58
Sam-I-Ami cant imagine trying to manage longer than 64s16:59
clarkbjeblair: ya it is an option if we need it but not our only option16:59
greghaynesjeblair: Yea, I mention it because IME it 'just works' if youre already using provider networking - all you do is create a new neutron network with the correct routing/network info16:59
mtreinishyay, looks like we're collecting all the gate things now: http://status.openstack.org/openstack-health/#/g/build_name/gate-cinder-python27?groupKey=build_name&resolutionKey=hour16:59
jeblair(doing our first ipv6-only region could be a real time sink -- a worthwhile one, but still one :)16:59
mtreinishdims: ^^^16:59
greghaynesjeblair: ah, right, client side16:59
Sam-I-Amarmax: one good thing about ovn is its being tested for scale/performance before getting out there :)16:59
yolandalet's see if we have more info about the move and the possibilities, before next week16:59
armaxmordred: as for the internal testing you guys are conducting, I am pretty sure you know above else how tricky it is to squeeze the most out of openstack with the gazillion knobs it has :)16:59
jeblairgreghaynes: like, the nodepool work for it should probably happen post-shade17:00
fungiSam-I-Am: but yeah, in my old service provider job since ipv6 rollout was happening under my group i encoded /48 per customer per datacenter into policy (we got arin to assign us a /28 initially so planned to start with /32 per data center)17:00
greghaynesjeblair: Yea, I totally spaced on the fact that client side might not 'just work'17:00
armaxSam-I-Am: that is good to hear17:00
clarkbnodepool already supports it iirc17:00
clarkbsomeone else with v6 only added the support17:01
*** ashleighfarnham has joined #openstack-infra17:01
fungihuawei i think?17:01
*** agentle has joined #openstack-infra17:02
Sam-I-Amfungi: cool. i worked at a service provider whose motto was if it doesnt work with v6, we dont use it... therefore, we v6'd thing a long time before anyone else.17:02
*** julim has joined #openstack-infra17:02
*** dizquierdo has quit IRC17:02
Sam-I-Amv6 since 200317:02
Sam-I-Amand its still not cool :/17:02
bauzasdo we have the meeting bot since today's netsplit ?17:02
*** sridhar_ram has joined #openstack-infra17:02
clarkbbauzas nick is 'openstack'17:02
SpamapSSam-I-Am: T-Mobile US is 100% v6 now.17:03
SpamapSSam-I-Am: AT&T and Verizon are also mostly v617:03
*** vgridnev__ has joined #openstack-infra17:03
bauzasclarkb: I can see it indeed17:03
SpamapSSam-I-Am: can't have 6 billion mobile phones on v4. ;)17:03
Sam-I-AmSpamapS: i wonder what bone-headed things they did17:03
Sam-I-Amlast time i was interacting with telcos and v6, they wanted to NAT everything17:03
Sam-I-Amand use longer than /6417:03
bauzasclarkb: for some reason, the topic name for the meeting is not yet updated, neither the discusison17:03
bauzas#openstack-meeting-317:04
clarkbmaybe it didnt identift17:04
clarkb*identify17:04
*** rockyg has quit IRC17:04
*** vgridnev__ has quit IRC17:04
SpamapSSam-I-Am: IIRC, Google mucked with Android's networking stack to not request a v4 address unless an app explicitly requires it, and then systematically declared war on apps that did it.17:04
fungiSam-I-Am: yep, we were providing v6 in ~2003 but unfortunately a lot of the backbone providers we were peering with still did not actually want to v6 peer with us17:05
SpamapSSam-I-Am: then they just do SNAT at the towers.17:05
clarkbSpamapS: android v6 is broken though as I discovered recently17:05
fungithat was honestly the hardest part. negotiating peering17:05
Sam-I-AmSpamapS: interesting17:05
clarkbit will attempt to internet via ula17:05
openstackgerritMerged openstack-infra/project-config: networking-ovn: Add install job for kuryr  https://review.openstack.org/27810117:05
Sam-I-AmSpamapS: not sure what my iphone does... it doesnt appear to ue 617:06
*** jlanoux_ has quit IRC17:06
SpamapSIt's inevitable in the next 10 years. People carrying a phone, a tablet, and with a car that has a hotspot.. all online all at the same time and not aware of eachother.17:06
SpamapSSam-I-Am: yeah, Apple is.. spehshul17:06
Sam-I-Amv6 was supposed to be end-to-end17:06
Sam-I-Amnone of this nat crap17:06
Sam-I-Amand then telcos happened17:06
SpamapSthe telcos aren't nearly as big a problem as the web hosts17:06
SpamapStelcos are "getting it" because they have no other choice17:07
Sam-I-Ami find it sad that my home isp offers v6... except its just a stupid tunnel that the dsl modem terminates17:07
fungiSpamapS: i'm curious what 100% means in t-mobile... i have a t-mobile wap/usb nic and only seem to be able to get an ipv4 address from it17:07
SpamapSbut anyway... v6 only region seems like a nice thing to have. ;)17:07
*** e0ne has quit IRC17:07
fungiSpamapS: purchased only a couple months ago, i might add17:07
Sam-I-AmSpamapS: i'm sure a lot of interesting neutron things will come to light with v6 only17:07
* mtreinish checks his phone now17:07
SpamapSfungi: every phone gets a v6 address according to recent claims they made.17:07
Sam-I-Amor even ANY v6 testing17:07
openstackgerritIan Cordasco proposed openstack-dev/pbr: Include wsgi_scripts in generated wheels  https://review.openstack.org/28027017:07
fungiSpamapS: might be, but this phone-like device on their network only seems to _forward_ ipv4, so for my purposes they're still 0% ipv6 :/17:08
Sam-I-Ammaybe its some crazy 4-in-6 business17:08
*** infra-red has joined #openstack-infra17:08
zaromorning17:08
fungino dhcp6, no v6 ra/nd17:08
Sam-I-Amyolanda: thx17:08
yolandaSam-I-Am anytime17:09
mtreinishlooks like my tmobile phone only gets a v617:09
SpamapSit's the phones17:09
SpamapSapparently it's all android phones17:10
SpamapSthat run marshmallow17:10
SpamapSwhich is like, 10 phone models ;)17:10
mtreinishthat's my phone :)17:10
SpamapSyep17:10
clarkbjust beware androind v6 isnt great17:12
clarkbit makes some really odd route decisions17:12
greghaynesclarkb: I blame your openwrt ula config ;)17:12
*** EricGonc_ has joined #openstack-infra17:13
clarkbworking v6 from tmobile and wifi, working v4 from tmobile and wifi, nvermind that send all traffic out ula interface!17:13
*** apoorvad has joined #openstack-infra17:13
greghayneswell, ula tends to be a pretty low-latency network17:14
*** ddieterly has quit IRC17:15
*** tphummel has quit IRC17:15
mtreinishI don't have v6 setup on my home network, so not too much of a concern yet. One day I'll bother to set it all up17:15
*** yamahata has joined #openstack-infra17:15
*** matrohon has quit IRC17:15
*** waynr has quit IRC17:16
*** waynr has joined #openstack-infra17:16
*** EricGonczer_ has quit IRC17:16
clarkbmtreinish: it usually mostly works but there are rough edges all over17:16
*** nt has quit IRC17:16
clarkblike openwrt's default config being bad for android which makes poor decisions17:17
Sam-I-Ami find a good chunk of my traffic goes over v6 now17:17
fungimtreinish: so if you configure your phone as a wireless hotspot, do clients connecting to it get ipv6 addresses?17:17
Sam-I-Amfacebook, google, youtube, etc.17:17
mtreinishfungi: that's a very good, question. Let me check17:17
fungimtreinish: i'm going to bet they end up on an rfc-1918 network with nat17:18
SpamapSMy AT&T Nexus 6 is also only supporting ipv6 at the moment17:18
clarkbI should say openwrt + luci not sure if luci or openwrt sets that default17:18
fungimtreinish: and get no ipv6 assigned at all17:18
*** jlibosva has quit IRC17:19
*** wolverineav has joined #openstack-infra17:19
*** mikelk has quit IRC17:20
mtreinishfungi: well ifconfig shows my laptop on the phone hotspot has an rfc1918 addr for v4 but there is also a v6 addr set17:20
funginon-linklocal?17:21
*** nt has joined #openstack-infra17:21
cody-somervillemtreinish: phone hotspots typically do SNAT.17:22
*** sabeen3 has quit IRC17:22
mtreinishfungi: dunno, I still ahven't learned how to read a v6 addr17:22
mtreinishfungi: http://paste.openstack.org/show/487307/17:22
mtreinishbut, yeah IIRC that's a link local addr17:22
clarkbon phone I have NAT'd v4 and proper v617:22
fungimtreinish: yeah, that's linklocal chosen by your machine. `ip -6 ro sh` would be more telling17:23
fungimtreinish: that would tell you whether you have any v6 routes at all17:23
*** tphummel has joined #openstack-infra17:24
fungiyou're looking for one starting with the word "default"17:24
*** wolverineav has quit IRC17:24
fungiso maybe `ip -6 ro sh|grep ^default`17:24
mtreinishfungi: just has a single line, no default: http://paste.openstack.org/show/487308/17:24
mtreinishso doesn't look like it gives a proper v6 on the hotspot17:25
*** sabeen1 has joined #openstack-infra17:25
fungiyep, that's your linklocal route for talking to other v6 addresses on the local lan17:25
*** shashank_hegde has joined #openstack-infra17:25
zaroyolanda: thanks for testing https://review.openstack.org/#/c/259157/17:25
zaroclarkb: ^ you interested in approving?17:26
clarkbjeblair at $.01/minute $25 is 41.6 call hours, that seems about right for a handful of people dialing did all day (doing maths to figure out how much we should guestimate paying on that number)17:26
*** scheuran has quit IRC17:26
jeblairclarkb: yes, and when i checked we had 5 pstn users, so that means about 8 hours/day, which is what's scheduled for the midcycle17:27
*** ddieterly has joined #openstack-infra17:29
*** sabeen2 has joined #openstack-infra17:29
*** sigmavirus24 has quit IRC17:30
Shrewsmtreinish: your shade download_image patch looks great, just need a little consistency with the exceptions. thanks for the patch17:30
*** jordanP has quit IRC17:30
*** sigmavirus24 has joined #openstack-infra17:31
*** marcusvrn_ has quit IRC17:31
*** xgerman has quit IRC17:31
fungijeblair: is batch reviewing in gertty supposed to be able to set label votes? wonder if maybe i'm hitting gerrit acl issues...17:31
jeblairfungi: yes17:31
*** sabeen1 has quit IRC17:32
fungii only get the option to set a message when i hit 'r'17:32
fungii'll try narrowing down which projects/branches are causing this17:32
*** agentle has quit IRC17:32
*** xgerman has joined #openstack-infra17:32
*** sc68cal has joined #openstack-infra17:32
Shrewsfungi: fyi, do not use gitpython 1.0.2 with gertty. it causes bad things17:33
*** marcusvrn_ has joined #openstack-infra17:33
*** mnencia has quit IRC17:33
*** isviridov_away has quit IRC17:33
*** larainema has quit IRC17:33
*** dalgaaf has quit IRC17:33
*** stevelle has quit IRC17:33
mtreinishShrews: I just used TypeError, because that's what python does if the args are wrong17:33
mtreinishbut I can change it17:33
*** rackertom has quit IRC17:33
*** erlon has quit IRC17:33
*** sridhar_ram has quit IRC17:33
fungiShrews: thanks for the heads up. is there a cap/exclusion set for it in master already?17:33
*** mescanef has quit IRC17:33
Shrewsmtreinish: awesome, thx17:33
Shrewsfungi: i think jeblair set one17:33
*** dtardivel has quit IRC17:34
*** zxiiro has quit IRC17:34
*** zhenguo_ has quit IRC17:34
*** betherly has quit IRC17:34
*** vmbrasseur has quit IRC17:34
*** patrickeast_ has quit IRC17:34
*** Ng has quit IRC17:34
*** skath_ has quit IRC17:34
*** ameade has quit IRC17:34
*** teran has quit IRC17:34
*** GheRivero has quit IRC17:34
*** DuncanT has quit IRC17:34
openstackgerritMerged openstack-infra/openstackid: Implemented display param on authentication OIDC request  https://review.openstack.org/28136117:34
*** daemontool has joined #openstack-infra17:34
jeblairyeah should be17:34
fungiShrews: ahh, yep. GitPython>=0.3.2.RC1,<1.0.217:34
*** matt6434 has quit IRC17:34
Shrews\o/17:34
*** wendar has quit IRC17:34
*** darrenc has quit IRC17:34
*** Nakato has quit IRC17:34
fungiso i did not upgrade to 1.0.2 when i updated my virtualenv17:34
*** davidlenwell has quit IRC17:34
*** kuromagi has quit IRC17:34
*** mcarden has quit IRC17:34
*** zigo has quit IRC17:34
*** dimtruck has quit IRC17:34
*** njohnston has quit IRC17:34
*** mhu has quit IRC17:34
*** sigmavirus24 has quit IRC17:35
*** jamielennox has quit IRC17:35
*** khappone_ has joined #openstack-infra17:35
*** wendar has joined #openstack-infra17:35
*** _bmjen is now known as bmjen17:35
fungilooks like i have 1.0.1 in there according to pip freeze17:35
*** larainema has joined #openstack-infra17:35
*** hockeynut has quit IRC17:35
*** AJaeger has quit IRC17:35
*** mgagne has quit IRC17:35
*** krtaylor_ has quit IRC17:35
*** eglute_s has quit IRC17:35
*** khappone has quit IRC17:35
*** dguitarbite has quit IRC17:35
*** ryanpetrello has quit IRC17:35
*** JayF has quit IRC17:35
*** lbragstad_ has quit IRC17:35
*** mmedvede has quit IRC17:35
*** d34dh0r53 has quit IRC17:35
jeblairfungi: i just verified batch review and setting code review +/-1 works for me on 4 changes on openstack-dev17:35
*** darrenc has joined #openstack-infra17:35
*** vmbrasseur has joined #openstack-infra17:35
*** craige has quit IRC17:36
*** kuromagi has joined #openstack-infra17:36
fungijeblair: looks like it's a problem with the stable branch acl for cinder. cinder master is fine and glance stable branch changes also work17:36
*** dtroyer has quit IRC17:36
*** zigo has joined #openstack-infra17:36
*** mcarden has joined #openstack-infra17:36
*** Nakato has joined #openstack-infra17:37
*** matt6434 has joined #openstack-infra17:37
*** davidlenwell has joined #openstack-infra17:37
*** zxiiro has joined #openstack-infra17:37
fungiyep, http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/acls/openstack/cinder.config has it exclusive to cinder-stable-maint with no entries for registered users17:38
fungior change owners17:38
fungii'll fix17:38
*** zhenguo_ has joined #openstack-infra17:39
*** rackertom has joined #openstack-infra17:39
*** eglute has joined #openstack-infra17:39
*** stevelle has joined #openstack-infra17:39
*** dimtruck has joined #openstack-infra17:39
*** hockeynut has joined #openstack-infra17:39
*** mgagne has joined #openstack-infra17:39
jeblairClint: just so you know, i'm -1 on https://review-dev.openstack.org/5357  honestly, bordering on -2.17:39
*** lbragstad_ has joined #openstack-infra17:39
*** njohnston has joined #openstack-infra17:39
*** AJaeger has joined #openstack-infra17:39
*** ryanpetrello has joined #openstack-infra17:39
*** JayF has joined #openstack-infra17:39
fungikeystoneclient has a similarly busted acl17:39
*** craige has joined #openstack-infra17:39
*** betherly has joined #openstack-infra17:39
*** DuncanT has joined #openstack-infra17:39
*** d34dh0r53 has joined #openstack-infra17:39
*** krtaylor_ has joined #openstack-infra17:39
jeblair(i'm clearly way behind on my review-dev reviewing)17:39
*** dtroyer has joined #openstack-infra17:39
fungiand neutronclient17:39
*** sigmavirus24_awa has joined #openstack-infra17:40
*** dguitarbite has joined #openstack-infra17:40
*** mnencia has joined #openstack-infra17:40
fungiand novaclient17:40
*** mhu has joined #openstack-infra17:40
*** mescanef has joined #openstack-infra17:40
*** sigmavirus24_awa is now known as sigmavirus2417:40
jeblairno wonder no one reviews stable branch changes :)17:40
*** GheRivero has joined #openstack-infra17:41
Clintjeblair: thanks17:41
fungii'll just go ahead and fix any i find for now, but we should probably take a deeper dive looking for any which have exclusive overrides on code-review and workflow labels17:41
*** teran has joined #openstack-infra17:41
*** ameade has joined #openstack-infra17:41
*** skath_ has joined #openstack-infra17:41
*** max_lobur has quit IRC17:41
*** patrickeast_ has joined #openstack-infra17:41
*** vgridnev__ has joined #openstack-infra17:42
*** dalgaaf has joined #openstack-infra17:42
*** hashar has quit IRC17:43
*** fmccrthy has quit IRC17:43
*** erlon has joined #openstack-infra17:43
*** rlandy has quit IRC17:43
*** dtardivel has joined #openstack-infra17:43
*** pvaneck has joined #openstack-infra17:44
*** aspiers has quit IRC17:44
*** Jeffrey4l has quit IRC17:44
*** jamielennox has joined #openstack-infra17:44
*** fmccrthy has joined #openstack-infra17:45
openstackgerritAliaksandr Buhayeu proposed openstack/python-jenkins: Adds possibility to retrieve all jobs from the view specified  https://review.openstack.org/27975017:45
*** Sukhdev has quit IRC17:45
*** rlandy has joined #openstack-infra17:45
*** aspiers has joined #openstack-infra17:45
*** isviridov_away has joined #openstack-infra17:45
*** Ng has joined #openstack-infra17:46
*** claudiub has joined #openstack-infra17:46
*** _nadya_ has joined #openstack-infra17:47
mtreinishShrews: looking at exc.py should I use OpenStackCloudResourceNotFound if there are no images returned by the search_images() call?17:48
*** _nadya_ has quit IRC17:48
*** tykeal has left #openstack-infra17:48
*** shashank_hegde has quit IRC17:49
Shrewsmtreinish: yeah, that's probably the more correct one to use17:49
Shrewsi don't think we've been very consistent with its usage17:51
*** morgabra has quit IRC17:51
*** morgabra has joined #openstack-infra17:51
*** regXboi has joined #openstack-infra17:52
fungigah, it looks like https://review.openstack.org/280439 tanked non-core revirewing for all these repos17:52
*** agentle has joined #openstack-infra17:53
fungitonyb: ^ sorry, looks like i gave you some incomplete advice and then didn't review the change17:53
fungiwe should only be removing that in acls which aren't setting exclusiveGroupPermissions to some other groups for those refs17:54
fungianyway, i have to run for a bit. back in an hour or so to finish sorting all this out17:54
pleia2hmm, is this related to the 'Registered Users' group we deleted ruring the renames last week?17:54
jeblairfungi: (why should we remove them even in that case (other than they aren't strictly necessary))?17:54
jeblairfungi: (iow, are you suggesting we remove it in those cases just for cleanup, or is there a functional reason?)17:55
openstackgerritMatthew Treinish proposed openstack-infra/shade: Add a method to download an image from glance  https://review.openstack.org/27881617:55
fungijeblair: yeah, it was meant to be cleanup, but i think someone got the idea that it was a suitable workaround for the broken groups entries i fixed yesterday17:55
mtreinishShrews, SpamapS: ^^^17:55
*** max_lobur has joined #openstack-infra17:55
*** daemontool_ has joined #openstack-infra17:55
fungii mean, it would have worked around it, but the groups files were fixed before that change was approved17:55
fungiand the change was overreaching17:56
*** mmedvede has joined #openstack-infra17:56
fungianyway, back in a little while17:56
jeblairpleia2: sounds like maybe they were trying to work around that issue?  but i'm catching up -- what caused the group to be created?17:56
*** mmedvede_ has joined #openstack-infra17:56
pleia2I think it was some automatic gerrit thing, related to the upgrade, zaro?17:57
*** e0ne has joined #openstack-infra17:57
*** daemontool has quit IRC17:57
*** agentle has quit IRC17:57
*** derekh has quit IRC17:58
Shrewsmtreinish: +2. thx17:58
*** _sarob has joined #openstack-infra17:58
*** achanda has joined #openstack-infra17:59
*** _ody_ is now known as ody17:59
*** ody is now known as _ody17:59
*** _nadya_ has joined #openstack-infra18:00
*** edmondsw has quit IRC18:00
*** tphummel has quit IRC18:00
openstackgerritClark Boylan proposed openstack-infra/nodepool: Increase timeout to give mysql more time  https://review.openstack.org/28098818:00
*** achanda has quit IRC18:02
*** tphummel has joined #openstack-infra18:02
*** julim has quit IRC18:03
*** sridhar_ram has joined #openstack-infra18:03
*** _sarob has quit IRC18:03
*** kozhukalov has quit IRC18:04
*** SotK has quit IRC18:04
*** Zara has quit IRC18:04
zarojeblair: groups got created from subprojects adding 'Registered Users' group to their acls18:05
*** max_lobur has quit IRC18:05
clarkbjeblair: zaro because that group is no longer in the database so the db lookup failed causing jeepyb to treat that as a new group creation request18:05
*** jsavak has quit IRC18:05
*** andymaier has quit IRC18:05
*** julim has joined #openstack-infra18:05
zarojeblair: which made manage projects create a custom group called 'Registered User's instead of using the gerrit system group, 'global:Registered-Users'18:05
*** _sarob has joined #openstack-infra18:06
*** jsavak has joined #openstack-infra18:06
zarojeblair: this is the change that fixed the issue, https://review.openstack.org/#/c/248126/18:06
*** Swami has quit IRC18:06
*** shashank_hegde has joined #openstack-infra18:07
zaroan gerrit upstream changed recently merged that prevents it from happening on gerrit as well.18:07
*** woodster_ has joined #openstack-infra18:08
Shrewsclarkb: mysql using innodb i assume?18:08
clarkbShrews: yup18:08
Shrewsah, yeah. drops can take time, depending on things18:08
*** sarob_ has joined #openstack-infra18:08
clarkbShrews: basically occasionally we have a test timeout whating for databse drop to happen and all of my debug output doesn't really show any problems so want to test theory that sometimes 60s is just not enough18:08
clarkbif I get a job run that rails on the 10 minutes timeout I am not sure what we will do18:09
clarkbs/rails/fails/18:09
Shrewsclarkb: how big is the db?18:09
zarofungi: reverting that change should fix it all up correct?18:09
clarkbShrews: should be small, a handful of rows at most in ~4 tables18:09
*** esikachev has joined #openstack-infra18:10
Shrewsclarkb: eh, hrm. that should be done fairly quickly. possibly something is open, keeping a lock on the table and preventing the drop18:10
* Shrews waves hands wildly, in mordred-esque style18:10
clarkbShrews: that is what we thought but we started dumping innodb status, processlist and open tables and nothing looks locked18:11
*** _sarob has quit IRC18:11
*** SotK has joined #openstack-infra18:12
clarkboh good we shouldn't be restricting memory on any of our jenkins slaves now18:13
*** _nadya_ has quit IRC18:13
*** ccrouch has quit IRC18:13
*** BobBall is now known as BobBall_AWOL18:14
*** e0ne has quit IRC18:15
*** e0ne has joined #openstack-infra18:16
*** sabeen2 has quit IRC18:17
*** maishsk has joined #openstack-infra18:17
*** e0ne has quit IRC18:17
AJaegerzaro: I reviewed the change again - let's revert it. The "Registered users" was only for exclusive usage, so need to be readded18:18
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Revert "Remove Registered Users from seeting -1..1 in Code-Review"  https://review.openstack.org/28145318:19
pleia2zaro: good to know about the gerrit upstream change for the future18:19
*** sputnik13 has joined #openstack-infra18:19
zaroAJaeger: wfm18:19
*** sabeen has joined #openstack-infra18:20
AJaegerzaro: please double check the change...18:20
pleia2AJaeger: gave my +2 to get us back to limping, but I'm confused, should this be some other group?18:20
pleia2like global:Registered-Users ?18:20
pleia2zaro: ^^18:20
*** agentle has joined #openstack-infra18:21
AJaegerpleia2: check https://review.openstack.org/#/c/248126/4/jeepyb/cmd/manage_projects.py18:21
pleia2AJaeger: oh ok, thanks :)18:21
pleia2makes sense18:21
zaropleia2: yeah, 'group Registered Users' will point to  'global:Registered-Users' after fix https://review.openstack.org/#/c/248126/18:22
* pleia2 thumbs up18:22
AJaegerpleia2: thanks for double checking!18:22
zarobefore it was pointing to '$UUID Registered Users' UUID being the custom group id18:23
*** Swami has joined #openstack-infra18:23
pleia2zaro: understood, thanks for explaining18:23
*** ociuhandu has quit IRC18:24
*** jsavak has quit IRC18:25
jeblairfungi, pleia2, zaro, AJaeger: approved https://review.openstack.org/28145318:25
pleia2thanks jeblair18:26
*** notnownikki has quit IRC18:26
zaroAJaeger: just saw a slight nit there. otherwise looks good18:26
jeblairzaro: yeah, that's the cleanup fungi was suggesting (remove that group where exclusive isn't set)18:26
jeblairbut someone can make that in a followup change18:27
*** ayoung has joined #openstack-infra18:27
zarosounds good18:27
AJaegerzaro: but exclusive is set there, isn't it?18:27
fungijeblair: AJaeger: zaro: back now--thanks for picking that up!18:27
*** sabeen3 has joined #openstack-infra18:27
fungiand i agree, reverting that was the best measure for now18:27
AJaegerzaro: you mean: Removing "refs/heads/stable/" completely? Yes, let's do that - will you provide a followup change?18:28
zaroAJaeger: sure, can do.18:28
*** sabeen has quit IRC18:28
*** kushal has quit IRC18:29
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Fix tag publishing of server projects  https://review.openstack.org/27923818:30
AJaegerfungi, changed to use tags everywhere ^18:30
*** asettle has quit IRC18:30
fungiright, any acls which have the same permissions granted in refs/heads/stable/* as refs/heads/* are redundant since we got rid of any exclusiveGroupPermissions for stable-maint-core in the all-projects acl18:30
AJaegerzaro: https://review.openstack.org/#/c/281453/1/gerrit/acls/openstack/murano.config is the same18:31
AJaegerzaro: no, isn't - sorry18:31
*** sabeen3 has quit IRC18:32
fungijeblair: ooh, i can definitely see the additional gertty sync load from conflicts-with now that i have self-owned changes in nova, neutron, et cetera18:32
openstackgerritMerged openstack/requirements: Bump PyECLib global-requirement to >= 1.2.0  https://review.openstack.org/27894518:33
*** achanda has joined #openstack-infra18:33
jeblairfungi: yeah, things can spiral out a bit in those cases...18:33
jeblairby the way...18:33
jeblairzaro: were you working on testing gc methods for the git repos in gerrit?18:34
*** agentle_ has joined #openstack-infra18:35
clarkbjeblair: fungi https://review.openstack.org/#/c/273703/12 and https://review.openstack.org/#/c/226114/ should both be ready for the last push to merge (they fix a couple nodepool bugs) and then I can do one more nodepool restart later today when image builds are done18:35
clarkbthen the last thing needing a restart is oscc's update for swift stuff18:36
*** sdague has quit IRC18:36
*** asselin_ has joined #openstack-infra18:36
*** rcernin has quit IRC18:36
clarkbthe oscc change merged just needs a release? let me check if that has been done yet maybe we can do all in one restart18:36
openstackgerritKhai Do proposed openstack-infra/project-config: Remove redundant access configuration  https://review.openstack.org/28146118:37
zaroAJaeger: ^18:37
*** sdague has joined #openstack-infra18:37
clarkbnot yet released. mordred were you planning on doing an oscc release soon?18:37
fungiclarkb: cool, i should have representative test results for various python jobs across more than a dozen of our major server and client projects later today18:37
clarkbfungi: woot18:37
fungifor stable branches as well as master18:37
*** wolverineav has joined #openstack-infra18:38
AJaegerthanks, zaro18:38
fungithe 40+ do-not-merge changes are all with topic:bindep-testing18:38
clarkbfungi: if those turn out happy its just a matter of checking rax uploads once oscc fix is in and flipping jobs over ya?18:38
fungiclarkb: yep, basically a change to edit the majority of our tox-based job templates, followed by a lengthy jjb update that will need babysitting18:39
AJaegerregarding nodepool: There's a change by mordred for project config: https://review.openstack.org/140840 "Add elements for Infra servers" - is that something we want to include soon?18:39
*** agentle has quit IRC18:39
*** kushal has joined #openstack-infra18:39
*** infra-re_ has joined #openstack-infra18:40
clarkbAJaeger: I think that too depends on reliable rackspace uploads since te majority of our proper servers live in rax18:40
yolandaAJaeger, i depend on that change for infra cloud18:40
*** asselin__ has quit IRC18:40
clarkbthe exception being infra cloud ^18:40
*** ccrouch has joined #openstack-infra18:40
yolandai include infra element to install puppet, install keys, etc18:40
*** jsavak has joined #openstack-infra18:42
*** infra-red has quit IRC18:42
jeblairwell that's a blast from the past.18:43
jeblairoh, apparently i said i was going to email the infra list with a link to the nova git repo tar, but i never did that18:43
jeblairor at least, i have no recollection of it18:43
AJaegerdo we still need "refs/heads/release-*" ? I see that in murano ACL. It's redundant, isn't it?18:43
zarojeblair: are you referring to doing more test in regards to this change? https://review.openstack.org/#/c/265356/18:44
jeblairoh, ha! i did18:44
jeblairhttp://lists.openstack.org/pipermail/openstack-infra/2016-January/003640.html18:44
crinkleyolanda: I reworked https://review.openstack.org/#/c/231757/ to try to avoid depending on that, forgot to drop the depends-on18:44
*** edmondsw has joined #openstack-infra18:45
*** eliqiao_ has quit IRC18:45
zarojeblair: ahh, i thought you were fishing for interests from others?18:46
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest  https://review.openstack.org/23175718:46
zarojeblair: i was waiting to see if there would be any replies.18:46
zarosince it looks like no replies i will give it a go and see what i find.18:46
jeblairzaro: thanks :)18:47
openstackgerritKaitlin Farr proposed openstack-infra/project-config: Upgrade Castellan functional gate  https://review.openstack.org/28146418:47
clarkblots of ansible puppet fails due to The error was: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='puppetdb.openstack.org', port=8081): Max retries exceeded with url: /v3/commands18:47
clarkbfungi: ^ I recall you had debuggd puppetdb recently? I wonder if our server is just too small for the load?18:47
pleia2jeblair: heh, I tweeted about it too and hoped for some interest given interactions, alas https://twitter.com/pleia2/status/68557488187904819218:48
*** eliqiao_ has joined #openstack-infra18:48
fungiclarkb: hadn't seen that one yet. issues i'd dealt with so far were corrupt report files, duplicate report keys in postgres, and the thing that turned out to be ansible-inventory switching between nova instance uuid and instance name for everything in dfw18:49
jeblairpleia2: yeah, lots of people have opinions though!18:49
pleia2jeblair: indeed18:50
pleia2moar work, fewer opinions18:50
openstackgerritMerged openstack-infra/nodepool: Delete building images on startup  https://review.openstack.org/22611418:52
jeblairclarkb, greghaynes: both aprvd18:52
greghaynesjeblair: \O/18:53
jeblairsuper nitpicky comment on 2737018:53
jeblairsuper nitpicky comment on 27370318:53
jeblairself-evidently notaminusone :)18:53
clarkbgah merge failed18:53
clarkbso needs updating anyways18:53
jeblairoh, sorry, i guess i should have approved those in the other order :(18:54
*** dizquierdo has joined #openstack-infra18:54
clarkbfungi: 8081 is the puppetdb java process not an apache proxy18:55
clarkbso that rules out any proxy funny business18:55
*** baoli has quit IRC18:55
*** doug-fis_ has joined #openstack-infra18:55
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Cleanup some ACLs  https://review.openstack.org/28146518:56
AJaegerzaro, fungi, here's another cleanup of ACLs ^18:56
*** baoli has joined #openstack-infra18:56
*** esikachev has quit IRC18:57
*** rossella_s has quit IRC18:57
*** rossella_s has joined #openstack-infra18:58
*** bpokorny_ has joined #openstack-infra18:59
*** doug-fish has quit IRC18:59
clarkbgreghaynes: will you be able to rebase that change?19:00
*** doug-fis_ has quit IRC19:00
*** arxcruz has joined #openstack-infra19:01
*** wolverineav has quit IRC19:02
*** bpokorny has quit IRC19:02
*** amitgandhinz has quit IRC19:02
*** amitgandhinz has joined #openstack-infra19:03
*** eil397 has joined #openstack-infra19:03
*** doug-fish has joined #openstack-infra19:03
*** agentle has joined #openstack-infra19:04
*** _nadya_ has joined #openstack-infra19:05
*** asselin_ has quit IRC19:05
clarkbthe puppetdb doesn't appear to be very loaded19:06
*** rcernin has joined #openstack-infra19:06
*** njohnston has left #openstack-infra19:06
*** vgridnev__ has quit IRC19:07
*** doug-fish has quit IRC19:07
*** agentle_ has quit IRC19:08
*** asselin_ has joined #openstack-infra19:08
*** lyarwood has joined #openstack-infra19:09
*** baoli has quit IRC19:09
*** baoli has joined #openstack-infra19:10
*** vgridnev__ has joined #openstack-infra19:11
AJaegerzaro, thanks for reviewing the ACL cleanup. fungi, could you check the comments by zaro and myself - and educate me, please? change #28146519:11
jeblair281465 looks like it breaks gertty; i'm digging in19:11
fungifun, i'll wait to pull that up in gertty just yet :/19:12
jeblair(there's a file mode-only change in there)19:12
AJaegershould I pull that out as separate change?19:12
clarkbAJaeger: the other thing to look at is the all-projects acls which most(all) projects inherit19:13
*** doug-fish has joined #openstack-infra19:13
jeblairAJaeger: gertty was waiting on this, which merged, like, a year ago, so i think we can fix it now :) https://github.com/gitpython-developers/GitPython/pull/26619:14
AJaegerclarkb: I checked that in a few cases but if you notice something I missed, please tell me19:14
*** vgridnev__ has quit IRC19:14
AJaegerjeblair: great19:14
clarkbAJaeger: the only thing I can think of is if projects want to prevent groups with +2 in all-projects from having +2 by keeping those exclusive group perms on the stable branches19:15
*** sabeen1 has joined #openstack-infra19:15
fungiwhich really they shouldn't do, because that breaks, e.g., our ability to override things as project bootstrappers group19:16
openstackgerritMerged openstack/requirements: update constraints to cliff 1.17.0  https://review.openstack.org/28085319:17
*** bpokorny_ has quit IRC19:17
*** bpokorny has joined #openstack-infra19:17
AJaegerclarkb: let me check the history of these files if there's any special commit message comment19:17
*** dtardivel has quit IRC19:18
*** sabeen3 has joined #openstack-infra19:18
*** vgridnev__ has joined #openstack-infra19:18
*** vgridnev__ has quit IRC19:19
openstackgerritJames E. Blair proposed openstack/gertty: Fix diff crash on perm-only changes  https://review.openstack.org/28147619:19
jeblairfungi: ^19:19
jeblairit doesn't show you the perms, but it doesn't crash.19:19
fungithanks, grabbing19:20
*** _nadya_ has quit IRC19:20
*** sabeen1 has quit IRC19:20
AJaegerclarkb: I'm not seeing anything mentioned in commit messages19:21
greghaynesclarkb: I am currently nomsing, you can rebase or I can in a few19:21
clarkbgreghaynes: no rush need to wait for images regardless, and I am sorting out how to oscc release19:22
clarkblooks like ptl or delegate can propose the openstack/releases change19:22
clarkbso maybe I can propose it and if mordred ACKs it we can haz release19:22
*** vgridnev__ has joined #openstack-infra19:22
*** jpr has joined #openstack-infra19:22
AJaegerclarkb: yes, anybody can propose a release change19:23
clarkbAJaeger: the docs say only ptl or delegate can19:23
clarkbwhich is a process thing not a gerrit acl19:23
clarkbneed to sort out if I need to do anything with reno first19:23
* jeblair notes that we don't need an oscc release to start using our new clouds19:23
AJaegerclarkb: agreed on the process...19:23
*** Zara has joined #openstack-infra19:24
clarkbjeblair: correct, this is to get working rackspace image uploads19:24
jeblairclarkb: oh, what was that change?19:24
jeblairif you have it handy19:24
clarkbjeblair: https://git.openstack.org/cgit/openstack/os-client-config/commit/?id=dd1f03c597daf1dc422608ab8e2b0b6b78168a3f19:25
*** vgridnev__ has quit IRC19:25
jeblairclarkb: thanks!19:25
*** Thelo has quit IRC19:25
clarkbjeblair: in theory that hands swiftclient enough info to reup tokens when they expire without keystone session support19:25
clarkbjeblair: which should fix the problem we have where auth fails after a while19:25
AJaegerclarkb: regarding reno: you should not need to do anything for the release, release notes should have been added already.19:25
clarkbAJaeger: is the expectation that when you write a change you add something to the releasenotes?19:26
AJaegerclarkb: but it's a good idea that releasenotes exist for all important changes and add them in if misssing19:26
clarkbright it doesn't look like oscc has been doing that19:26
AJaegerclarkb: yes, expectation is that you take care of releasenote when you make a change19:26
AJaegerclarkb: ;(19:26
clarkbAJaeger: roger, so that hasn't happened but I can probably write some19:26
*** agentle has quit IRC19:27
clarkb(that is extra change round trip though)19:27
*** rguillebert has quit IRC19:27
jeblairdoes oscc have previous release notes anywhere?  i'm wondering what kind of things it would include in release notes...19:28
*** Thelo has joined #openstack-infra19:28
*** nijaba has joined #openstack-infra19:28
*** nijaba has joined #openstack-infra19:28
jeblairadding/removing vendors?  maybe?19:28
openstackgerritHenry Gessau proposed openstack/requirements: Constrain eventlet to 0.18.2 for now  https://review.openstack.org/28147919:28
asselin_hi, I'm running into a nodepool issue where launching nodes and deleting nodes is taking over 10 minutes.19:28
clarkbjeblair: I don't think it has anything like that outside of the git log19:28
*** abregman is now known as abregman|nb19:29
clarkbjeblair: reno was added to it before the last release but release notes haven't been added using reno from what I can tell19:29
*** piet has joined #openstack-infra19:29
asselin_I've added plenty of debug statements and starting to think it's a threading or database issue.19:29
*** claudiub has quit IRC19:29
asselin_Anyone have tips to further diagnose those?19:29
clarkbasselin_: I want to say our experience has been that is the cloud api being slow19:29
AJaegerclarkb: So, add a releasenote for the one change you need the release for ;)19:30
*** daemontool_ has quit IRC19:30
clarkbasselin_: you might try listing your servers outside of nodepool and see if that takes a significant amount of time19:30
jeblairasselin_: can you paste some debug log lines?19:30
*** MarkAtwood has quit IRC19:30
*** daemontool_ has joined #openstack-infra19:30
asselin_jeblair, e.g. http://paste.openstack.org/show/487323/19:31
jeblairasselin_: can you paste a lot more lines than that? :)19:32
*** vgridnev__ has joined #openstack-infra19:32
asselin_jeblair, sure :)19:32
jeblairasselin_: seeing how the providermanager is dealing with tasks would help evaluate clarkb's theory19:32
*** aorourke is now known as aorourkeaway19:32
asselin_jeblair, I added a ton of debugging trying to test clarks' theory.19:33
clarkbAJaeger: ya reading reno docs now to see how that is done, I wonder if I should try and add a bunch more19:33
clarkbAJaeger: for this release at least19:33
*** fawadkhaliq has quit IRC19:33
*** Sukhdev has joined #openstack-infra19:33
asselin_jeblair, here's a slice http://paste.openstack.org/show/487324/19:34
jeblairasselin_: what's the cpu/usage like of nodepoold ?19:34
AJaegerfungi, there are origin/release-0.1 branches on murano - shall I add that section back in?19:34
asselin_19564 nodepool  20   0 1356m  92m 5164 S    2  4.6   4:54.20 nodepoold19:34
*** dtantsur is now known as dtantsur|afk19:35
asselin_  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND19:35
asselin_19564 nodepool  20   0 1356m  92m 5164 S    1  4.6   4:55.51 nodepoold19:35
*** electrofelix has quit IRC19:35
jeblairasselin_: i don't see the word 'task' in that log at all19:35
*** Zara is now known as Zara_19:35
*** agentle has joined #openstack-infra19:35
*** baoli has quit IRC19:35
*** baoli has joined #openstack-infra19:36
*** Zara has joined #openstack-infra19:36
openstackgerritSam Betts proposed openstack-infra/project-config: Make Ironic VM memory and count configurable  https://review.openstack.org/28148419:36
*** ccrouch1 has joined #openstack-infra19:36
jeblair2016-02-17 14:55:48,037 DEBUG nodepool.ProviderManager: Manager bluebox-sjc1 running task ListFloatingIPsTask (queue: 0)19:37
jeblair2016-02-17 14:55:48,332 DEBUG nodepool.ProviderManager: Manager bluebox-sjc1 ran task ListFloatingIPsTask in 0.294368028641s19:37
asselin_jeblair, http://paste.openstack.org/show/487326/19:37
jeblairasselin_: you should have lines like that ^19:37
*** Zara_ has quit IRC19:37
openstackgerritKaitlin Farr proposed openstack-infra/project-config: Upgrade Castellan functional gate  https://review.openstack.org/28146419:37
*** ddieterly has quit IRC19:37
*** vgridnev__ has quit IRC19:38
asselin_jeblair, yeah, noticing that now....good info to consider19:38
fungiAJaeger: either that or find out if they want those branches deleted19:38
AJaegerfungi, origin/release-0.5 is the newest murano branch and last updated in May 201419:38
*** vgridnev__ has joined #openstack-infra19:38
AJaegerfungi: let me add it back for now...19:38
jeblair2016-02-17 19:24:07,471 DEBUG nodepool.ProviderManager: Manager mpc ran task <nodepool.provider_manager.AddFloatingIPTask object at 0x7f210c52e3d0> in 43.8285300732s19:39
jeblairasselin_: ^19:39
*** ccrouch has quit IRC19:39
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Use SSL for rabbitmq traffic  https://review.openstack.org/27637519:39
fungiAJaeger: yeah, the only reason i asked was because they were using their murano-milestone group for stable branches and also for release-* branches, so that section was not redundant with refs/heads/*19:39
fungiwhere they were using murano-core instead19:39
AJaegerfungi: yeah19:40
asselin_jeblair, yeah....need to figure out how that adds up to 10+ minutes. Will look at those in more detail.19:40
jeblairasselin_, clarkb: it looks like adding a floating ip can take 1.5 to 1 minute19:40
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Cleanup some ACLs  https://review.openstack.org/28146519:40
AJaegerfungi: updated ^19:40
AJaegerthanks for review and comments!19:40
*** achanda has quit IRC19:40
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Cleanup some ACLs  https://review.openstack.org/28146519:41
jeblairasselin_: i see quite a few floating ip actions thant are 20-60 seconds, and since you're spinning up >10 servers there, could feasibly add up19:41
asselin_jeblair, oh...so they're sequential?19:41
asselin_ok I see now19:41
jeblairasselin_: yes.  we have found that results in marginally fewer complete public cloud outages.  :)19:42
*** esikachev has joined #openstack-infra19:42
asselin_you mean nodepool takes down a cloud?19:42
*** armax has quit IRC19:42
jeblairasselin_: more than once.19:42
jeblairrunning on my desktop at home.19:42
*** baoli has quit IRC19:42
asselin_I guess my cloud is taking revenge19:43
*** zeih has joined #openstack-infra19:43
jeblair(and not the same provider either)19:43
asselin_jeblair, is there an easy way to increase the threads to e.g. 5?19:43
*** ccrouch1 has quit IRC19:43
*** vgridnev__ has quit IRC19:44
*** baoli has joined #openstack-infra19:44
*** ccrouch has joined #openstack-infra19:45
fungiasselin_: this is also why we have built-in tunable api call throttling per provider in nodepool19:45
jeblairasselin_: it may not make as much of a difference as you expect -- many of these things end up being sequential in openstack itself due to queue processing/database locks, etc.  but, yes, you can approximate it by defining multiple providers in nodepool.19:45
fungidifferent clouds get angry at different rates19:45
asselin_thunder19:46
*** ddieterly has joined #openstack-infra19:46
fungiserializing certain calls or adding in delays rather than letting nodepool request things at the rate it would really prefer has been pretty necessary19:46
jeblairasselin_: we had 5 hpcloud providers (but not for this reason (reason:neutron limitations))19:46
* asselin_ looks up tunable throttling api19:46
*** lucasagomes is now known as lucas-dinner19:46
*** piet has quit IRC19:46
fungii think it's called "rate"?19:46
*** wolverineav has joined #openstack-infra19:47
jeblairyes, and i think it defaults to one api call per second19:47
openstackgerritMerged openstack-infra/project-config: Revert "Remove Registered Users from seeting -1..1 in Code-Review"  https://review.openstack.org/28145319:47
jeblairas soon as an api call takes longer than a second, you've already passed beyond the point at which adjusting the rate is going to help19:47
fungiwhich is a conservative default. we turn it down to a much smaller delay on most of ours19:47
clarkbAJaeger: https://review.openstack.org/281489 how does that look?19:47
jeblairfungi: indeed19:47
*** tzn has quit IRC19:47
asselin_rate19:48
asselin_In seconds. Default 1.0.19:48
fungithough the name of the parameter is sort of backwards. known cosmetic issue19:48
fungishould really be called "delay" or something19:48
jeblairyeah, it's inverted19:48
fungibecause a lower value results in a higher rate and vice versa19:48
asselin_rate: 0.0519:48
jeblairasselin_: is that what you have it set to?19:48
asselin_yes...good or bad?19:49
fungiso that's basically waiting 0.05 seconds between calls, or shooting for ~20 calls/second19:49
asselin_(not sure who set it..just what it is now)19:49
jeblairasselin_: it's fine.  it's basically not affecting you19:49
fungithat might be the puppet-nodepool or puppet-openstackci default19:49
AJaegerclarkb: reads fine but let's see how it renders...19:49
jeblairasselin_: your provider's fastest response time is 0.2 seconds, so you are already running serially as fast as possible.19:50
*** karlamrhein has joined #openstack-infra19:50
asselin_jeblair, the other strange issue is that it seems to have start a few days ago....maybe something changes in the cloud19:50
*** mriedem has quit IRC19:51
jeblairasselin_: possibly.  we've also seen cases where things get slowly (or even exponentially) worse over time as database tables fill up, etc19:51
asselin_nodepool database?19:51
jeblairin general, i think most of the cases where we identified that were fixed in openstack, but...19:51
jeblairasselin_: no, openstack's database19:51
jeblairasselin_: are you operating this cloud?19:52
asselin_jeblair, no, a different team is...will have to ask them19:52
jeblairasselin_: yeah, maybe they can track down why adding a floating ip can take a whole minute...19:52
*** ashleighfarnham has quit IRC19:53
jeblairasselin_: maybe it's something they can see in logs, but maybe it's something where they would need to watch the database while nodepool is running19:53
asselin_ok thanks will follow up with that19:53
*** sdake has quit IRC19:54
clarkbShrews: you might be interested in https://review.openstack.org/281489 as I have tried to record one of the changes you wrote iwth mordred in the release notes19:55
*** sdake has joined #openstack-infra19:57
fungiwow, really? H104 needed to be a rule?19:57
fungi"H104  File contains nothing but comments"19:58
clarkbthe license part I can kind of see, the no comments is wat19:59
*** sdake has quit IRC19:59
fungiyeah, my "test" changes basically all add a do-not-merge.py which contains just a comment line, because so many projects now don't trigger the majority of their jobs unless you modify a python file19:59
*** sdake has joined #openstack-infra19:59
fungibut it looks like if i want pep8 results i'll need to put some no-op python in there20:00
*** tzn has joined #openstack-infra20:00
fungior i can just ignore the fact that pep8 will fail, since i need to check the logs anyway20:00
*** _nadya_ has joined #openstack-infra20:00
AJaegerfungi: I think pep8 always runs20:00
clarkb# This will show hacking\assert True20:00
fungiAJaeger: right, but i also wanted to run unit tests20:01
fungiso anyway, this is working for my needs, it's just reminding me that the world is full of silly rules lawyers20:01
AJaegerfungi: got it20:01
*** ccrouch has quit IRC20:01
jeblairfungi: openstack is a rules lawyer honeypot20:02
fungiall my changes ended up with a -1 from jenkins, and hacking was why20:02
*** ccrouch has joined #openstack-infra20:02
fungijeblair: it's a many things honeypot, for sure20:02
*** vgridnev__ has joined #openstack-infra20:02
*** apoorvad has quit IRC20:02
*** jed56 has quit IRC20:03
*** rfolco has quit IRC20:04
fungijeblair: once the gerrit acl revert is applied, how quickly should i expect gertty to reflect that? does it check permissions on sync? only when there's a new patchset?20:05
*** karlamrhein has quit IRC20:06
*** sarob_ has quit IRC20:08
*** aysyd has quit IRC20:08
jeblairfungi: on sync; you can ctrl-r it to update20:09
jeblair(when you're looking at a change where you suspect the perms are wrong)20:09
fungijeblair: yep, just wondering if that reevaluates all permissions when it does so20:09
fungisounds like yes20:09
jeblairyes it should20:09
fungiawesome--thanks!20:09
openstackgerritKhai Do proposed openstack-infra/git-review: WIP: push change with a vote  https://review.openstack.org/28150420:09
*** |-paul-| has joined #openstack-infra20:10
*** infra-red has joined #openstack-infra20:10
fungii will admit with some embarrassment that there are times when my impatience gets the better of me and i ctrl-r changes to see if they have updates without waiting for gertty to catch up with its overall sync20:10
*** mriedem has joined #openstack-infra20:11
fungithe patch a while back that made it prioritize ctrl-r on a change over background sync is really helpful for my impatience20:11
*** infra-re_ has quit IRC20:13
pleia2btw, the ubuntu wikis are being hit pretty hard as well by spammers via launchpad logins (just like ours), I've been tracking any launchpad-side changes they may look into, but ultimately they are using a different wiki tech than us (moinmoin vs mediawiki) so wiki-side solutions will differ20:13
pleia2afaik no one is actively tackling this though20:14
*** agentle has quit IRC20:14
pleia2the ubuntu folks have turned their wiki read-only20:14
clarkbpleia2: on our side jp volunteered to take a look on the mailing list, not sure where that has gotten to20:14
pleia2clarkb: nods, maybe I should follow up to see20:15
* pleia2 does so20:15
jeblairfungi: me too -- also note that if you use the 'remote' feature we added to follow links from terminals, it auto-syncs that change20:15
* nibalizer has a smallish wiki that is also under assault20:15
nibalizerI don't think it has anything to do with lp logins though20:15
*** gordc has quit IRC20:15
jeblairfungi: and i think that's true for most "open this one specific change directly" actions20:16
fungivery cool20:16
*** ldnunes has quit IRC20:16
openstackgerritIan Cordasco proposed openstack-dev/pbr: Include wsgi_scripts in generated wheels  https://review.openstack.org/28027020:16
fungiclarkb: oh cool, that was in fact jp! for some reason i thought jimmy volunteered (he and jp may be dopplegangers, but they are still in theory separate sentient life forms)20:18
fungiclarkb: btw he's in here as jpmaxman20:18
fungino idea if he has any channel highlight triggers20:19
pleia2oh good, I'm replying on list anyway o/ jpmaxman20:19
jpmaxmanHi :)20:19
fungii guess that answers my question! ;)20:19
openstackgerritKaitlin Farr proposed openstack-infra/project-config: Upgrade Castellan functional gate  https://review.openstack.org/28146420:19
stevemarquestion about one of the requirements jobs20:20
stevemarthe output of the gate-requirements-pep8 job in this patch https://review.openstack.org/#/c/281028/1 was not so helpful :)20:20
*** vgridnev__ has quit IRC20:21
stevemar"File Not Found"20:21
jpmaxmanSorry been meaning to follow-up on wiki.20:21
stevemarcan't tell if it's something i did.. don't want to do yet another recheck20:21
pleia2jpmaxman: thanks, feel free to reach out if you need help getting familiar with our infra20:22
pleia2jpmaxman: like "I know how to fix this, help me write the patch" is ok :)20:22
jpmaxmanYes I will. Need to take a few and look at it. Will try to do it this afternoon.20:22
jpmaxmanGreat :)20:22
pleia2wonderful20:22
*** kushal has quit IRC20:23
AJaegerstevemar: runtime of pep8 was 30mins - so that was a timeout...20:23
AJaegerstevemar: still, it should have a log file20:23
stevemarAJaeger: yeah, 30 minutes was the suspicious part :)20:24
fungithanks jpmaxman!20:25
zaroanybody know the secret to running git-review tests?  i can't seem to get it to work.20:25
clarkbzaro: how do they fail?20:25
fungizaro: i would work backwards form the job definition, but i seem to recall it being relatively workable (i was able to test it on my workstation a little while back)20:26
zaroclarkb: No such file or directory: '/home/ubuntu/git-review/.gerrit/golden_site'20:26
fungiit should create that20:26
clarkbbut maybe it doesn't craete it if it thinks it is already there?20:26
clarkb(not sure why it would think that if the file doesn't exist)20:27
zaro.gerrit is there but not golden_site folder20:27
zarodo you need to run gerrit locally?20:27
clarkbzaro: no, it runs gerrit locally for you20:28
*** dprince has quit IRC20:28
*** vgridnev__ has joined #openstack-infra20:28
*** vgridnev__ has quit IRC20:28
zaroyeah, that's what i thought.20:28
*** achanda has joined #openstack-infra20:28
*** vgridnev has joined #openstack-infra20:28
*** esker has joined #openstack-infra20:29
*** zeih has quit IRC20:29
*** daemontool__ has joined #openstack-infra20:29
fungiright, the job sets up multiple gerrits if memory serves20:31
*** sabeen3 has quit IRC20:32
fungiand that golden site is the original state it uses for each of them before they diverge for per-test setup20:32
fungii need to look back at it though... been a while20:32
*** daemontool_ has quit IRC20:32
*** vgridnev has quit IRC20:32
*** _nadya_ has quit IRC20:32
AJaegerfungi, could you quickly review https://review.openstack.org/#/c/281465/ again, please? I removed the murano change and updated commit message only...20:33
*** krotscheck is now known as krotscheck_dcm20:34
zaroclarkb, fungi : no java in my VM20:34
*** mkj_wk has quit IRC20:34
*** imcsk8 has quit IRC20:35
fungiaha, that would do it20:35
*** rajinir has joined #openstack-infra20:36
*** jsavak has quit IRC20:36
*** dizquierdo has quit IRC20:36
tonybthorst: Yeah I basically wanted to lurk in the channel to get a feel for how you guys are going.20:37
thorsttonyb: awesome.  Lurk away  :-)20:37
tonybthorst: It's very low impact :)20:37
*** jsavak has joined #openstack-infra20:38
*** sergmelikyan has joined #openstack-infra20:38
*** sabeen has joined #openstack-infra20:38
sergmelikyanI have quick questions regarding proposal bot20:38
sergmelikyanHow often it is running? How soon you can expect update from global-requirements proposed to your project?20:38
clarkbsergmelikyan: updates to projects are triggered when changes that produce updates merge20:39
fungiassuming it's not broken20:39
clarkbsergmelikyan: so it is all related to the amount of changes happening to global requirements20:39
clarkbfungi: right20:39
AJaegerpabelanger: could you review a grafana change, please? https://review.openstack.org/#/c/27883220:39
*** mkj_wk has joined #openstack-infra20:39
pabelangerAJaeger: sure20:39
AJaegersergmelikyan: why do you ask?20:39
sergmelikyanAJaeger: I have this change-request https://review.openstack.org/278983 merged 27h ago20:41
sergmelikyanand we are waiting this change propagated to murano requirements - around 10 commits are blocked by this20:41
sergmelikyanwe always can make it manually but I decided to check why we don't see commits from proposal bot20:42
AJaegersergmelikyan: let's check latest run: https://jenkins.openstack.org/job/propose-requirements-updates/763/consoleFull20:42
*** sabeen3 has joined #openstack-infra20:42
AJaegerwait, that's liberty20:43
*** agentle has joined #openstack-infra20:43
clarkbhttps://git.openstack.org/cgit/openstack/murano/tree/requirements.txt#n23 looks up to date to me?20:44
openstackgerritBrant Knudson proposed openstack-infra/project-config: Keystone-only uwsgi job  https://review.openstack.org/26499120:45
*** sabeen has quit IRC20:45
sergmelikyanclarkb, AJaeger sorry guys20:45
AJaegersergmelikyan: this merged 23 hours ago https://review.openstack.org/#/c/280860/20:45
sergmelikyan:( Looks like I somehow missed that and didn't check actual requirements.txt20:45
pabelangerclarkb: had somebody ask the other day about multinode setup for centos7 and why we don't run one.  I assume it was because nobody stepped up to maintain it?20:46
sergmelikyanthought that if I didn't see commits they didn't happen20:46
clarkbpabelanger: ya there hasnt really been a need20:46
clarkbpabelanger: for the  work that was done either or are fine and I prefer ubuntu :)20:47
*** amrith is now known as _amrith_20:47
pabelangerclarkb: right, that is what I suspected.20:47
fungibasically, for the jobs that currently run on multi-node ubuntu trusty, would any of them test something substantially different on multi-node centos 7 (beyond what's already tested on all-in-one centos 7 based jobs)?20:47
clarkbfungi: no20:47
fungiright20:48
clarkbthe only possible difference that occurs to me is libvirt versions and live migration20:48
fungiso the main impetus for multi-node centos 7 would be for things which aren't testable on ubuntu20:48
fungiwhich would likely be new jobs20:49
pabelangerYa, the only real stuff being asked about centos, was ensuring configuration options worked over multinode.  eg: not always binding to localhost20:49
fungirather than just centos 7 versions of existing jobs20:49
openstackgerritBrant Knudson proposed openstack-infra/project-config: Keystone-only uwsgi job  https://review.openstack.org/26499120:49
clarkbpabelanger: but the same config is used on centos and ubuntu for say nova right?20:49
pabelangerand stuff related to more centos coverage in general20:49
pabelangerclarkb: correct20:50
clarkbright so you don't actually cover that20:50
fungi"more centos coverage" sounds like distro image pettiness in disguise20:50
clarkbor rather it is already covered20:50
*** gtmanfred is now known as gtmanfred-rebell20:50
*** gtmanfred-rebell is now known as gtmanfred20:50
pabelangerthis was more for testing like puppet / ansible to deploy over centos / fedora.  Rather then functional / unit testing20:50
fungiin theory we're testing openstack, we're not testing how well ubuntu or red hat does openstack20:51
*** _sarob has joined #openstack-infra20:51
*** ddieterly has quit IRC20:51
pabelangerRight, the context of the conversation was could some of this be moved upstream, or continue testing midstream / downstream20:51
fungibut yes, for config management/orchestration frameworks which have a lot of distro-specific logic branching, i can see that being useful20:51
clarkbpabelanger: single node should cover that just fine? trying to understand hwree the gap is that multinode on centos would fill20:52
*** ddieterly has joined #openstack-infra20:52
funginot necessarily milti-node centos being useful for that, just centos testing in general20:52
*** aorourkeaway is now known as aorourke20:52
pabelangerclarkb: I think it was more for complex deployments, like HA and such.  I don't have all the details for the use case, it just started out with why is there not centos multi-node setup20:52
*** matt6434 is now known as mattoliverau20:53
*** gordc has joined #openstack-infra20:53
*** jungleboyj has joined #openstack-infra20:53
*** _sarob has quit IRC20:53
jungleboyjfungi: I hear you are the person to talk to about ATC passes for the summit.20:53
clarkbya for that there simply wasn't a need, we were testing that openstack could run multiple compute hosts with one also being a controller and have functioning live migration and shelve and networking20:54
*** _sarob has joined #openstack-infra20:54
fungijungleboyj: depends on what you mean by "talk to about" but i am the one who pushes the button/throws the switch on the mass mailer20:54
clarkbthis didn't require a specific distro, we went with what we had and were comfortable with20:54
*** rguillebert has joined #openstack-infra20:54
fungijungleboyj: i explicitly don't make policy decisions, grant exceptions, et cetera. for those sorts of things you want to contact summit@openstack.org20:55
clarkbpabelanger: at the time I think centos6 was the option not 7 too fwiw20:55
openstackgerritSean McGinnis proposed openstack/requirements: Update to os-brick 1.0.0  https://review.openstack.org/28152320:55
fungiright, and our centos 6 based testing was driven primarily by a need for some platform with a native python 2.6 base20:55
pabelangerclarkb: For sure20:56
notmynamecan I -2 the "DO NOT MERGE" patches so they aren't on my dashboard? will that negatively affect anyone?20:56
jungleboyjfungi: Not looking to complain.  I am an ATC and have not gotten a code.  Wondering if you can check if I am on the list and resend if possible.20:56
*** ddieterly has quit IRC20:56
*** esker has quit IRC20:56
funginotmyname: i'm trying to wip them all, but we have a lot of incorrect acls, mostly for stable branches20:56
funginotmyname: i'll go ahead and wip the ones which are bothering you while i work through what other projects have done to make their acls so much worse20:56
notmynamefungi: ah. I can do that too, I think20:56
*** kzaitsev_mb has joined #openstack-infra20:57
AJaegerfungi: swift uses "label-Workflow = -1..+1 group swift-stable-maint"20:57
notmynamefungi: do you mind if I workflow -1 those patches?20:57
AJaegerfungi: there're quite a few projects where WIP is disabled for change owner on stable branches ;(20:57
funginotmyname: feel free. i just did a workflow -1 on the master change but i lack permissions to wip (or even abandon) my own changes on swift's stable branches20:58
notmynameok20:58
*** dalgaaf has quit IRC20:58
fungiAJaeger: yeah, i'm trying to figure out if this is something i can/should solve through our normalization script or just hack together by hand20:58
*** esker has joined #openstack-infra20:58
notmynamefungi: done20:59
fungithanks notmyname!20:59
*** ericksonsantos has quit IRC20:59
*** infra-re_ has joined #openstack-infra20:59
crinklenotmyname: you could add `NOT message:"DO NOT MERGE"` to your dashboard query21:00
pc_mAJaeger: & Infra folks... With https://review.openstack.org/#/c/280952/1 it is (now) failing the docs-constraints jobs, but when I try to look at logs it says File Not Found. Any ideas?21:00
clarkbso here is a question, this release of oscc will remove hpcloud and runabove which is a non backward compat change. Do we 2.0 oscc? I am leaning towards no because those clouds don't work anyways21:01
AJaegerpc_m: runtime is 30mins - so that's a timeout.21:01
clarkbeg it wasn't our backward incompatible change21:01
AJaegerBut it should have created a log file nevertheless ;(21:01
fungicrinkle: yep, though ultimately i'd like to fix our stable branch acls so that change owners can be expected to wip changes like that (i was intending to do so had gerrit let me)21:01
* clarkb goes with 1.15.0 instead of 2.0.021:02
fungibecause if i don't fix this, i'll also be unable to abandon many of these once i'm done testing with them21:02
*** infra-red has quit IRC21:02
AJaegerfungi: I'd love to see that change21:02
jungleboyjfungi: Should I just e-mail summit@openstack.org to get help?21:02
fungijungleboyj: what e-mail address(es) were you using for your gerrit account? i can check what the query shows and whether i attempted to send you one21:03
*** apoorvad has joined #openstack-infra21:03
*** esker has quit IRC21:03
*** amitgandhinz has quit IRC21:03
pc_mAJaeger: Any suggestions as to what I should do?21:03
AJaegerpc_m: recheck21:03
pc_mAJaeger: ok. will try21:03
*** _sarob has quit IRC21:03
jungleboyjfungi: jsbryant@us.ibm.com  Thank you very much for looking!21:04
AJaegerpc_m, team: I wonder why this happens. This is the second change in hte last few hours21:04
*** amitgandhinz has joined #openstack-infra21:04
fungijungleboyj: i tried to send one to you on january 13. let me check my list of bounces21:04
*** e0ne has joined #openstack-infra21:04
fungijungleboyj: i never got a bounce/ndr back from that address21:05
fungijungleboyj: i can resend it if you want21:05
jungleboyjfungi: Weird.  Could you please?  I would appreciate it.21:05
AJaegerjungleboyj: did you check your spam?21:05
*** apoorvad has quit IRC21:06
jungleboyjAJaeger: Who would it have come from?21:06
fungijungleboyj: yeah, it has a few urls in it, and is sent from a cloud provider's network, so could end up marked as spam21:06
*** apoorvad has joined #openstack-infra21:06
clarkbmordred: https://review.openstack.org/281526 I hav eproposed that for oscc, if you are ok with it would be good to ACK it21:06
clarkbmordred: makes a 1.15.0 release21:06
AJaegerjungleboyj: "Jeremy Stanley"21:06
fungijungleboyj: sender is jeremy@openstack.org (that's the foundation e-mail address i basically don't use for anything except summit invites and vulnerability announcements)21:06
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Fix on display type touch  https://review.openstack.org/28152721:07
jungleboyjAJaeger: Nope, it if got marked SPAM it was before it got to me.21:07
jungleboyjAJaeger: I blame Lotus Notes.21:07
* jungleboyj hopes you will pitty me.21:07
*** jsavak has quit IRC21:07
* AJaeger hugs jungleboyj21:08
smcginnisjungleboyj: I pity you.21:08
*** jsavak has joined #openstack-infra21:08
* AJaeger waves good-bye21:08
jungleboyjAJaeger: Thank you.  I needed that.  :-)21:08
*** AJaeger has quit IRC21:08
*** angdraug has joined #openstack-infra21:09
openstackgerritMerged openstack-infra/subunit2sql: Add DB API method to get failure rate for jobs by run_metadata  https://review.openstack.org/26550721:13
fungijungleboyj: i've sent you another copy just now21:14
openstackgerritHenry Gessau proposed openstack/requirements: Constrain eventlet to less than 0.18.0 for now  https://review.openstack.org/28147921:14
fungijungleboyj: let me know if you don't receive this one21:14
jungleboyjfungi: You rock!  I just got it.  Thank you so much for the help!21:15
fungijungleboyj: no worries, glad it was an easy fix--i'd hate to not see you in austin!21:15
clarkbjamielennox: are you around yet? for credential logging (passwords/tokens) how safe is it to make logs available publicly if we remove debug logs?21:15
*** ddieterly has joined #openstack-infra21:15
jungleboyjfungi: I will be there.  Had hotel setup but realized I needed to actually get into the summit.21:16
fungihah21:16
*** esikachev has quit IRC21:18
*** bkero has quit IRC21:18
*** bkero has joined #openstack-infra21:18
*** Sukhdev has quit IRC21:18
jamielennoxclarkb: there shouldn't be any authentication requests logged even on debug21:19
jamielennoxclarkb: for tokens we were SHA1-ing them and printing that21:20
clarkbjamielennox: and this assumes keystoneauth or also keystoneclient?21:20
jamielennoxi would need to check we're still doing the SHA1 thing but passwords shouldn't end up in debug21:20
jamielennoxclarkb: should be both21:20
jamielennoxclarkb: have an example of otherwise?21:21
clarkbjamielennox: I don't just trying to understand if this is still a consideration21:21
clarkbjamielennox: good to hear that it shouldn't be21:21
fungiwith my vmt hat on, the current recommendation is still not to trust redaction features at debug level. though truth is keystone has been better about that for longer than a lot of other services21:26
clarkbya if we publish anything I think we should still publish > DEBUG21:27
clarkbbut that does open the doors to possibly publishing logs for things like nodepool and infra cloud21:27
fungiand the others are catching up to the point where i'm willing to switch to treating info leaks at debug level as an actual vulnerability if the rest of the vmt agrees21:28
jeblairi think we decided to set up a public elk for infra-cloud21:28
jeblairi support just doing that and if we run into problems, fix 'em and change the keys :)21:28
fungiat least in the past year i think we've had at least as many sensitive information leaks in non-debug logs as we have at only debug level, which is way better than openstack used to be21:29
fungiat least inasmuch as what was reported21:29
*** kgiusti has left #openstack-infra21:30
fungiokay, time for me to stop saying "at least" now21:30
clarkbhttps://jenkins04.openstack.org/job/gate-devstack-bashate/955/console21:30
clarkbmore network connectivity to git.o.o problems? rax-iad intsead of ovh21:30
fungiclarkb: rackspace has network maintenance going on this week21:31
fungiso possibly related21:31
jeblairfungi: at least you didn't say it again21:39
fungiheh21:40
*** ashleighfarnham has joined #openstack-infra21:40
*** thorst has quit IRC21:40
*** thorst has joined #openstack-infra21:41
*** armax has joined #openstack-infra21:41
*** karlamrhein has joined #openstack-infra21:41
*** achanda has quit IRC21:42
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Handle dib image deletion during periodic cleanup  https://review.openstack.org/27370321:42
*** thorst_ has joined #openstack-infra21:42
*** abregman|nb is now known as abregman21:45
*** thorst has quit IRC21:45
tristanCfungi: agreed, but probably starting at mitaka then, last time I checked, my kilo deployment still writes token in debug logs...21:45
openstackgerritSpencer Krum proposed openstack-infra/zmq-event-publisher: Update links in pom.xml  https://review.openstack.org/28154021:46
fungitristanC: yeah, maybe we should revisit after kilo-eol21:46
*** maishsk_ has joined #openstack-infra21:46
tristanCbut if projects agrees it's not acceptable to log token/password in debug (unless a non-default option such as "dont_hide_secrete_in_debug" is used), then yes, that sounds like a good idea!21:46
*** thorst_ has quit IRC21:47
*** maishsk has quit IRC21:47
*** maishsk_ is now known as maishsk21:47
fungii'm pretty sure we've wanted to get there for a long time, it's just that for so much of openstack's history the amount of care taken for debug logging was simply not there21:48
*** |-paul-| has quit IRC21:49
fungito the point that we had so many known examples of it, the vmt excluded them from vulnerability criteria just to keep its workload reasonable21:49
jeblairyikes21:50
fungisort of like the current situation with lack of ssl certificate validation in server-to-server communication21:50
*** samuelBartel has joined #openstack-infra21:50
*** dkranz has quit IRC21:51
fungionce an overwhelming majority of server-to-server ssl/tls actually provides some mechanism to verify its not being mitm'd, we can start considering outliers as actual vulnerabilities21:51
fungibut for now, much like the "don't publish debug logs" recommendation, we just stick to "you need to be able to trust all the systems on your management network" line21:54
jheskethMorning21:56
clarkbfungi: is that due to underlying python ssl not checking?21:56
*** abregman has quit IRC21:56
clarkbfungi: the whole insecure platform warning from requests21:56
fungiclarkb: some of it, yes21:56
*** karlamrhein has quit IRC21:57
fungiclarkb: some is also just flat disabling validation and not providing configuration options to be able to enable it21:57
clarkbwow21:57
*** imcsk8 has joined #openstack-infra21:57
fungiin part wit the excuse that python didn't provide a mechanism to solve it in the general case anyway21:57
fungiso the "plug in the ca you want to use to validate rest api calls between services" bit was left as a non-urgent future todo21:58
*** thorst has joined #openstack-infra22:00
*** sabeen3 has quit IRC22:01
openstackgerritKirill Zaitsev proposed openstack/requirements: Bump python-muranoclient to a 0.8.2 mitaka version  https://review.openstack.org/28154422:02
fungiand things like, your cinder driver is going to talk to your san over http to do provisioning, oh but some sites deploy this brand of san in an https-only configuration, okay well optionally let it communicate via https and maybe some day someone will solve the part where you can even make sure it's really your san and not some imposter22:02
*** dkranz has joined #openstack-infra22:03
fungi(https as a protocol, not as a security measure)22:03
*** salv-orl_ has joined #openstack-infra22:03
*** jtomasek is now known as jtomasek|afk22:04
*** thorst has quit IRC22:04
notmynamein launchpad, which is the group that has permissions for bug triage (eg setting priority, etc). is it drivers or maintainers?22:04
mgagneBug supervisor? :D22:05
*** sdake has quit IRC22:05
funginotmyname: yeah, as mgagne says, whichever group is configured as the bug supervisor. if there isn't one, i think it falls to the project owner/maintainer22:06
notmynamefungi: ok, thanks22:06
*** salv-orlando has quit IRC22:06
mgagnedriver: https://help.launchpad.net/Projects/Registering#Roles_within_projects22:06
*** shashank_hegde has quit IRC22:06
notmynameand how does that list get updated, eg when the list of cores changes? is it automatic or does it need to be updated by hand?22:06
funginotmyname: so for example https://bugs.launchpad.net/swift shows over in the right-hand column that "Swift Bug Team" is the supervisor22:07
mgagnemaintainer is the fallback22:07
mgagnedepends, for nova, it's an open team iirc22:07
mgagnehttps://launchpad.net/~nova-bugs22:07
mgagnefor swift, it's moderated22:07
*** KimJ has joined #openstack-infra22:07
notmynameactually, I'm looking at https://launchpad.net/python-swiftclient22:08
fungiyep, looks like no bug team is set there22:08
mgagnehttps://launchpad.net/~swift-drivers is a restricted team amd https://launchpad.net/~swift-bugs is moderated22:08
*** rhallisey has quit IRC22:08
mgagnehttps://bugs.launchpad.net/python-swiftclient Bug supervisor: Swift Bug Team22:09
fungid'oh. i thought i was looking at the bugs page22:09
fungiso tes, same swift bug team22:09
*** infra-re_ has quit IRC22:09
notmynamewhere do you see that?22:09
fungilooks like mordred maybe created that group in 2010 and is still the only admin for it?22:09
mgagnejust follow the links and check Membership policy22:09
funginotmyname: https://bugs.launchpad.net/python-swiftclient22:09
mgagne^22:10
fungiin the right-hand column22:10
notmynameI don't see a "swift bug team" anywhere22:10
mgagnebecause bugs tracking can be disabled in LP so you might not have a bug team at all22:10
fungithere's a link to Swift Bug Team under where it says Bug supervisor22:10
mgagnesee right sidebar22:10
funginotmyname: bugs.launchpad.net not launchpad.net22:10
notmynamethe string "bug supervisor" isn't on that webpage (for me)22:11
fungiin case you missed that nuance in the url22:11
notmynameah22:11
notmynameyeah, I did22:11
mgagneI checked while logged out too and it's still showing22:11
*** eliqiao_ has quit IRC22:11
notmynameok, so that basically resolves to swift-coresec22:12
notmynameand how is that updated? by hand, or does it get synced from gerrit groups?22:12
mgagneno sync22:12
fungiby hand22:12
mgagnehuman hands are used22:12
notmynameok, thanks22:12
*** regXboi has quit IRC22:12
notmynamethat's what I needed to know22:12
*** bpokorny has quit IRC22:13
*** eliqiao_ has joined #openstack-infra22:13
*** karlamrhein has joined #openstack-infra22:13
*** dkranz has quit IRC22:15
funginotmyname: i've switched it to you as the administrator of https://launchpad.net/~swift-bugs now too22:15
fungiso you can directly add/remove others from that group22:16
*** jsavak has quit IRC22:16
notmynamefungi: ok, thanks22:16
*** erlon has quit IRC22:16
fungithere was no need for mordred to be the admin of that group, 6 years later ;)22:17
openstackgerritBen Nemec proposed openstack/diskimage-builder: Revert "Correct rhel-common for rhel6"  https://review.openstack.org/28155122:17
notmyname:-)22:17
*** marcusvrn_ has quit IRC22:17
*** ybathia has joined #openstack-infra22:17
*** _sarob has joined #openstack-infra22:20
*** karlamrhein has quit IRC22:20
zaroclarkb: wow, git-review tests take ~40mins to complete?22:21
fungizaro: sounds about right22:21
*** _sarob has quit IRC22:21
ianwyay -> /dev/xvda1      301G  9.1G  279G   4% / <- f23 disk size issues fixed!22:21
*** jaosorior has quit IRC22:21
*** _sarob has joined #openstack-infra22:21
*** _sarob has quit IRC22:21
fungiianw: congrats!22:22
*** daemontool__ has quit IRC22:22
*** _sarob has joined #openstack-infra22:22
* fungi would #success that one22:22
*** jaosorior has joined #openstack-infra22:22
*** _sarob has quit IRC22:22
fungiit's certainly been plaguing us for weeks22:22
*** _sarob has joined #openstack-infra22:23
*** _sarob has quit IRC22:23
ianwthere's an outstanding change in the next dib release to fix symlinking of /var/run to /run, but other than that, for the first time in a *long* time, i'm not aware of any issues22:23
*** ddieterly has quit IRC22:24
greghaynesianw: good thing we decided to stop writing bugs. Things got a lot better since then. ;)22:26
ianwgreghaynes: ha ha :)  there's something up with centos-minimal and dependencies i haven't quite root caused yet.  so there's still things to do :)22:26
greghaynesAw22:27
ianwgreghaynes: speaking of breaking things -> the name changes in https://review.openstack.org/#/c/211859/22:27
*** e0ne has quit IRC22:27
*** ddieterly has joined #openstack-infra22:27
*** sdake has joined #openstack-infra22:27
greghaynesianw: yea, that should be all backwards compatible though22:28
ianwi'm just not sure "packageinstall" is really any more intrinsically understandable than "minimal"22:28
greghaynesI hipe22:28
greghaynesianw: its different - minimal is minimal but package install is meant to be a useful image (with things like sshd). Name suggestions welcome, though.22:29
*** alivigni has quit IRC22:30
greghaynesIdeally -package install would be the main Ubuntu/Fedora element but backwards compat22:30
*** ybathia has quit IRC22:31
*** sabeen has joined #openstack-infra22:32
*** doug-fish has quit IRC22:33
*** sabeen3 has joined #openstack-infra22:34
*** doug-fish has joined #openstack-infra22:34
*** doug-fish has quit IRC22:34
*** gildub has joined #openstack-infra22:34
*** doug-fish has joined #openstack-infra22:34
*** _sarob has joined #openstack-infra22:35
*** roxanaghe has left #openstack-infra22:35
openstackgerritMerged openstack-infra/openstackid: Fix on display type touch  https://review.openstack.org/28152722:35
*** sabeen has quit IRC22:36
*** Sukhdev has joined #openstack-infra22:38
jeblairminimal; minimally useful; useful; actually useful; bloated.22:39
jeblairwhich one includes emacs is left as an exercise for the reader (<3 emacs)22:40
openstackgerritMatt Riedemann proposed openstack-infra/project-config: Enable gate-tempest-dsvm-neutron-src-oslo.service for liberty  https://review.openstack.org/28155922:42
*** tiswanso has quit IRC22:42
*** ybathia has joined #openstack-infra22:44
*** EricGonc_ has quit IRC22:44
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Keep Gerrit ACL lines deduplicated  https://review.openstack.org/28156222:45
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Unshadow All-Projects in exclusiveGroupPermissions  https://review.openstack.org/28156322:45
fungiAJaeger will hopefully be pleased by that ^22:45
*** julim has quit IRC22:46
fungii know i at least am looking forward to being able to wip and abandon my test changes on stable branches22:47
*** mriedem has quit IRC22:49
*** jpr has quit IRC22:50
greghaynesJeb22:50
greghaynesjeblair: they all include different emacs versions22:51
jeblairgreghaynes: ;)22:51
*** baoli has quit IRC22:51
*** armax has quit IRC22:51
*** baoli has joined #openstack-infra22:52
*** amitgandhinz has quit IRC22:53
*** jungleboyj has quit IRC22:54
yolandanibalizer,around? i wanted to talk with you about the need of afs mirrors on infra cloud22:54
*** baoli has quit IRC22:54
*** baoli has joined #openstack-infra22:55
*** esker has joined #openstack-infra22:56
*** achanda has joined #openstack-infra22:56
*** doug-fish has quit IRC22:57
*** akuznetsova has quit IRC22:57
*** doug-fish has joined #openstack-infra22:57
nibalizerhi22:57
*** rossella_s has quit IRC22:57
nibalizerwhats up?22:57
*** rossella_s has joined #openstack-infra22:58
nibalizeras i understand it: we need mirrors23:00
yolandahi, so i was spinning a test nodepool and connected to infra cloud east23:00
nibalizerbut the procedure for that is pretty well understood23:00
*** dteselkin has quit IRC23:00
*** baoli has quit IRC23:00
*** xyang1 has quit IRC23:00
yolandathen i found that ready_node scirpt, configure-mirror, was failing. And Monty told that we need to have an AFS mirror on each cloud23:00
yolandai created a task for it on storybard, my suggestion for simplicity was to create the afs mirror, colocated in baremetal23:01
*** igormarnat has quit IRC23:01
yolandawhat do you think?23:01
jeblairyolanda: in your test nodepool, just disable the ready script23:01
crinklei thought we had already gotten nodepool to work without a mirror?23:01
yolandajeblair yes i did that23:01
*** dingyichen has joined #openstack-infra23:01
yolandabut is a temporary solution right?23:01
yolandathat's feedback i got from mordred23:02
jeblairyou don't need it for testing -- instead, you can just run devstack/tempest and look at the time it took to run tests to check performance23:02
*** doug-fish has quit IRC23:02
*** mmedvede_ has quit IRC23:02
jeblairyolanda: if we're ready to add it to production nodepool, yes, we need a mirror, but we don't want to run it on bare metal23:02
yolandajeblair, we are on that step now. We have a test nodepool, jenkins... and i'll start launching nodes there23:02
*** e0ne has joined #openstack-infra23:02
jeblairyolanda: you have a test jenkins?23:02
yolandajeblair i spinned one today23:02
jeblairyolanda: you don't need that either23:03
yolandabut i'm having trouble with gearman23:03
jeblairyolanda: none of that needs testing23:03
yolandajeblair, i wanted to test the whole workflow of nodepool. How do i better test it?23:03
jeblairyolanda: you are testing nodepool, not the cloud23:03
*** tzn has quit IRC23:03
*** dteselkin has joined #openstack-infra23:03
*** sarob_ has joined #openstack-infra23:03
SpamapSso, jjb.. talk to me about 'builder'. If a builder fails, does that make the job just retry?23:03
*** sarob_ has quit IRC23:03
jeblairSpamapS: no it fails the build23:03
*** shardy has quit IRC23:04
*** sarob_ has joined #openstack-infra23:04
*** akuznetsova has joined #openstack-infra23:04
SpamapSah ok, bash to the rescue then. :)23:04
*** baoli has joined #openstack-infra23:04
*** tzn has joined #openstack-infra23:04
SpamapSjeblair: thank you :)23:04
jeblairyolanda: i think running a test nodepool to build and destroy a vm is all you need to do23:04
fungiSpamapS: and it short-circuits, skipping any of the remaining builders23:04
*** _sarob has quit IRC23:04
*** igormarnat has joined #openstack-infra23:04
fungiSpamapS: though it still fires the publishers23:04
jeblairyolanda: at least, all you need to do to test that the cloud is ready for nodepool23:04
fungiSpamapS: which is why we do log collection in a publisher23:04
yolandajeblair i've done that. But i wanted to test performance, run jobs23:04
SpamapSahhhh23:05
SpamapSfungi: got it23:05
jeblairyolanda: okay, you don't need to set up jenkins on it.  just run devstack.23:05
rcarrillocruzso23:05
jeblairyolanda: if you want, you can grab reproduce.sh from any recent devstack/tempest run, and it will reproduce that exact run on your cloud23:05
rcarrillocruzafs mirror23:05
yolandajeblair, so far i found only one problem: https://review.openstack.org/#/c/281310/23:05
rcarrillocruzvm instead of metal23:05
rcarrillocruzwe have consensus here?23:05
yolandaproblem with deleting vms due to incorrect call to floating ips23:05
rcarrillocruzcrinkle: jeblair, nibalizer23:05
jeblairrcarrillocruz: yes, we have 2 accounts in every cloud region.  one is for long running servers, the other is for nodepool.23:06
rcarrillocruzi was thinking of collocating with bifrost, but don't have a strong opinion on this23:06
yolandajeblair thanks for the advice, whenever i needed to test a cloud, i did with the jenkins and zuul setup, so it might be easier23:06
jeblairrcarrillocruz, yolanda: the mirror goes in the account for long running servers ("openstackci" is the name we use)23:06
rcarrillocruzokidoki23:06
*** tpsilva has quit IRC23:06
jeblairyolanda: i promise, running "./reproduce.sh" is easier than setting up jenkins and zuul.23:07
yolandajeblair, the other alternative that was talked, is to create an independent project in our infracloud, then create a vm and setup the mirror on it23:07
*** tiswanso has joined #openstack-infra23:07
yolandajeblair, i'm pretty sure23:07
crinklei don't know what the requirements are for afs or why it would need to be colocated on the baremetal server23:07
yolandai didn't know about that reproduce23:07
*** tiswanso has quit IRC23:07
jeblairyolanda: what we're looking for at this point is does it *function*.23:07
jeblairyolanda: once it functions, we're ready to just add it to the production nodepool and collect further data that way23:08
yolandaok so after the patch i showed you, i can say it functions23:08
*** tiswanso has joined #openstack-infra23:08
jeblairyolanda: so, honestly, if you have spun up instances with nodepool, then we're ready to put it into production.  we'll set it to max-servers=1 at first to make sure it doesn't go badly, but can increase from there.23:09
jeblaircrinkle: the mirror has very few requirements; one sec.23:09
yolandajeblair my concern there, as i showed for west as well, is that is not clearly deployable. I have a mix of cherry-picks of code until everything lands23:09
rcarrillocruzcrinkle: it doesn't need...we could put it on vm or just in bifrost, since it's a host that we will barely use once we spin up all computes and controller23:10
rcarrillocruzbut pattern is vm23:10
jeblaircrinkle: http://specs.openstack.org/openstack-infra/infra-specs/specs/unified_mirrors.html#servers23:10
rcarrillocruzso i'll go with vm23:10
jeblairrcarrillocruz: yes, we very much want this to operate just like any other cloud23:10
nibalizerrcarrillocruz: but we put the mirrors on nova instances23:10
rcarrillocruz++, no snowflakes23:10
jeblaircrinkle: i didn't mention ram there23:11
nibalizerits how we work23:11
*** esker has quit IRC23:11
jeblaircrinkle: but i think we use 4g given a choice23:11
yolandathe concern i had with vms, is the problem of orchestration. Because it means you need to create another project, create a vm... if we have to redeploy the cloud for some reason, we'll loose that vm23:11
rcarrillocruzi can ansible that23:11
yolandagood thing is that it can be reproduced easily and doesn't contain any data that  need to persist23:12
*** sigmavirus24 is now known as sigmavirus24_awa23:12
jeblairyolanda: that's a really good point.  it is worth considering making it a snowflake for that purpose.  but maybe let's stick with the existing pattern for now and maybe talk about other options next week?23:12
*** gordc has quit IRC23:12
jeblairrcarrillocruz: that might be a good solution23:12
rcarrillocruzyeah23:12
jeblairyolanda: and yeah, with the new afs mirrors, they are really fast to set up; no initial data load needed23:13
jeblaircrinkle: even less than i thought: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=18223:13
*** esker has joined #openstack-infra23:13
jeblaircrinkle: that's what i made on bluebox, and is probably what we should do on omfra too23:13
*** dims_ has joined #openstack-infra23:13
*** krtaylor_ has quit IRC23:13
jeblair2g ram, 2vcpu, 200G of disk.23:14
*** dims has quit IRC23:14
crinklejeblair: I was just wondering what the motivation for putting it on the bifrost server was (which was not appealing to me), it sounds like we've moved past that23:14
jeblairi honestly think we're going to greatly reduce the amount of disk used, but we can do that after we're done adding content to the mirror.23:14
*** _sarob has joined #openstack-infra23:15
craigeo/23:15
jeblaircrinkle: yeah, certainly not for performance.  surviving cloud deploys might be a reason to consider, but if we can avoid it, i think it'd be worth it because it's so tiny (so i lean toward starting with vm and trying to work with that).23:16
*** xarses has joined #openstack-infra23:16
xarsesI thought we had the ability to "restore" etherpad from a revision23:17
*** esker has quit IRC23:18
xarsesI want to restore/promote https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda/timeslider#23266 back23:18
*** sdake_ has joined #openstack-infra23:18
*** sarob_ has quit IRC23:19
fungirelated point, etherpad is a terrible platform for evolving historical logs of information. it's useful for short-term collaboration on a document23:19
*** tiswanso has quit IRC23:19
fungibut if you keep editing and adding to it, at some point it will break or at best become extremely unwieldy23:20
*** sdake has quit IRC23:20
*** tzn has quit IRC23:20
fungii believe you can export the content from a particular point in time, if the history of the pad is not corrupted/lost23:21
fungibut would need to paste it back in and would lose authorship information23:21
fungiafaik, it doesn't work like a wiki where you can roll back the official version to an earlier edit23:22
xarsesdon't have to 'roll' it back, I just want all the content and authorship pushed back onto the end23:24
yolandaso leaving for today, it's quite late here. jeblair, thanks for the pointer on reproduce, i'll test it tomorrow23:24
rcarrillocruzyep23:24
rcarrillocruzme too23:24
rcarrillocruzcrashing now23:25
pleia2night rcarrillocruz23:25
rcarrillocruzg'nite23:25
jeblairyolanda, rcarrillocruz: thanks!  good luck23:25
xarsesMy colleague and I swear there used to be some option in the history view that would allow to "restore" it23:25
pleia2and yolanda23:25
fungixarses: i don't understand the difference between what you're asking for and rolling back (restoring) the content to a previous edit23:25
*** agentle has quit IRC23:25
fungiit's possible that was a feature of etherpad and was never implemented in etherpad-lite?23:25
xarsesfungi: aparently, it's bad to remove the edits from the histroy, and it's not allowed unless you are an admin23:26
fungii don't remember it, but that doesn't mean it never existed23:26
*** achanda has quit IRC23:26
*** sc68cal has quit IRC23:27
*** e0ne has quit IRC23:27
xarsessupposedly we can export / inport but I don't see how I do that in the current namespace23:27
fungiyeah, you could export and then paste it back into the current pad, but regardless exports don't include authorship/colors23:27
fungiit's just the textual content23:27
*** sergmelikyan has quit IRC23:28
xarsesimport fails23:28
*** ddieterly has quit IRC23:28
xarsesit requires a new page23:28
fungiimport requires a new page, yes23:29
xarseswhich I'm guessing loses the authorship23:29
fungideleting all the content from the pad and pasting the text from the export back in place of it doesn't require a new pad23:29
fungihowever, pasting huge amounts of text into a pad is also twitchy23:29
*** achanda has joined #openstack-infra23:33
*** sdake_ has quit IRC23:35
*** dimtruck is now known as zz_dimtruck23:36
*** baoli has quit IRC23:36
*** agentle has joined #openstack-infra23:37
*** agentle has quit IRC23:38
*** patrickeast_ is now known as patrickeast23:39
*** sdake has joined #openstack-infra23:41
*** armax has joined #openstack-infra23:42
*** mriedem has joined #openstack-infra23:43
*** Qiming has joined #openstack-infra23:43
*** bpokorny has joined #openstack-infra23:44
*** Sukhdev has quit IRC23:45
*** sdake_ has joined #openstack-infra23:45
*** rajinir has quit IRC23:45
*** sdake has quit IRC23:46
xarsesso the page is now fugly23:47
xarsesawesome23:47
*** edmondsw has quit IRC23:48
*** harlowja_at_home has joined #openstack-infra23:49
*** aeng has quit IRC23:52
*** _sarob has quit IRC23:52
*** pahuang has quit IRC23:53
*** rbrndt has quit IRC23:53
*** alex_xu has quit IRC23:53
fungiyeah, that's what i meant by "twitchy." you lose a lot of formatting when pasting. etherpad doesn't deal with pasting whitespace (indentation, blank lines, et cetera)23:53
funginor bullets, ordered lists, styles23:54
fungibut you can at least get the basic text content23:54
*** aeng has joined #openstack-infra23:54
*** alex_xu has joined #openstack-infra23:55
*** ybathia has quit IRC23:55
*** tphummel has quit IRC23:56
*** sc68cal has joined #openstack-infra23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!