Monday, 2016-02-01

ianwgreghaynes: ok, cool.  i'm still thinking installing from git will work, will finish up that idea today my time, might be one to disucss in meeting00:00
*** chlong has quit IRC00:01
*** thorst has joined #openstack-infra00:02
*** dims_ has joined #openstack-infra00:02
*** salv-orlando has quit IRC00:03
*** thorst has quit IRC00:04
*** thorst has joined #openstack-infra00:04
*** dims has quit IRC00:05
openstackgerritMatthew Thode proposed openstack/diskimage-builder: fix gentoo hardened support  https://review.openstack.org/27379500:05
prometheanfireianw: think that's all of them00:05
prometheanfireyep, nothing left to do is hope gate is working today and wait for reviews00:07
*** iremizov has quit IRC00:08
*** tpeoples has quit IRC00:08
*** kgalanov has quit IRC00:08
*** evgenyl has quit IRC00:08
*** mwhahaha has quit IRC00:09
*** iremizov has joined #openstack-infra00:09
*** evgenyl has joined #openstack-infra00:09
*** mwhahaha has joined #openstack-infra00:10
*** kgalanov has joined #openstack-infra00:10
*** tpeoples has joined #openstack-infra00:10
*** ihrachys has quit IRC00:10
*** GheRivero has quit IRC00:12
*** dham1 has quit IRC00:12
*** GheRivero has joined #openstack-infra00:12
*** dham1 has joined #openstack-infra00:12
*** thorst has quit IRC00:13
*** betherly has quit IRC00:14
*** DuncanT has quit IRC00:15
*** betherly has joined #openstack-infra00:15
*** evgenyl has quit IRC00:16
*** DuncanT has joined #openstack-infra00:17
*** evgenyl has joined #openstack-infra00:17
*** chlong has joined #openstack-infra00:17
*** fedexo has quit IRC00:22
*** Sukhdev has joined #openstack-infra00:28
openstackgerritMatthew Thode proposed openstack/diskimage-builder: add new cloud-init element  https://review.openstack.org/27376400:29
openstackgerritMatthew Thode proposed openstack/diskimage-builder: package-installs doesn't work in python3/gentoo  https://review.openstack.org/27378200:33
jlvillalAny idea how often the CI bot runs to generate a new upper-constraints.txt file?00:34
*** ihrachys has joined #openstack-infra00:36
*** woodster_ has quit IRC00:36
prometheanfireianw: anything else?00:37
*** elo has joined #openstack-infra00:38
ianwprometheanfire: thanks, i'll loop back around this afternoon my time and take another look00:38
prometheanfireianw: k, hoping gating is reliable this time mainly :P00:39
ianwprometheanfire: no, the ironic job seems to have gone haywire again :(00:39
prometheanfirelol, yay00:39
*** ihrachys has quit IRC00:39
prometheanfireso maybe sometime next month00:40
prometheanfireby that I don't mean tomorrow00:40
*** Anticimex has quit IRC00:44
*** markvoelker has joined #openstack-infra00:46
*** Anticimex has joined #openstack-infra00:48
*** salv-orlando has joined #openstack-infra00:49
*** markvoelker has quit IRC00:50
*** hogepodge has quit IRC00:56
*** hogepodge has joined #openstack-infra00:59
*** kzaitsev_mb has quit IRC01:00
*** ihrachys has joined #openstack-infra01:01
*** markmcclain has quit IRC01:02
*** Jeffrey4l_ has joined #openstack-infra01:02
*** dims_ has quit IRC01:04
*** markmcclain has joined #openstack-infra01:05
openstackgerritMerged openstack-infra/project-config: Add devstack-centos7 to internap and blue box  https://review.openstack.org/27416101:07
*** HenryG_ is now known as HenryG01:08
*** thorst has joined #openstack-infra01:11
*** camunoz has joined #openstack-infra01:11
*** thorst has quit IRC01:18
*** salv-orlando has quit IRC01:23
*** zhenguo_ has joined #openstack-infra01:25
*** mriedem is now known as mriedem_away01:42
*** markmcclain has quit IRC01:43
*** ashleighfarnham has joined #openstack-infra01:44
*** markmcclain has joined #openstack-infra01:45
*** salv-orlando has joined #openstack-infra01:45
*** markvoelker has joined #openstack-infra01:47
*** calebb has joined #openstack-infra01:47
*** ashleighfarnham has quit IRC01:49
*** markvoelker has quit IRC01:52
*** boris-42 has quit IRC01:53
*** yamamoto has joined #openstack-infra01:54
*** tonytan4ever has joined #openstack-infra02:00
*** yamamoto has quit IRC02:00
*** baoli has joined #openstack-infra02:02
*** dims has joined #openstack-infra02:02
*** baoli has quit IRC02:04
openstackgerrithardik proposed openstack-infra/project-config: gate-mistral-docker-buildimage has been moved to post  https://review.openstack.org/27448602:05
*** zhurong has joined #openstack-infra02:07
*** dims has quit IRC02:12
*** tonytan4ever has quit IRC02:12
*** dims has joined #openstack-infra02:12
*** tonytan4ever has joined #openstack-infra02:13
*** dims has quit IRC02:13
*** thorst has joined #openstack-infra02:16
*** salv-orlando has quit IRC02:17
*** tonytan4ever has quit IRC02:18
*** tonytan4ever has joined #openstack-infra02:18
*** tonytan4ever has quit IRC02:19
*** tonytan4ever has joined #openstack-infra02:19
*** changzhi has joined #openstack-infra02:20
*** Sukhdev has quit IRC02:22
*** thorst has quit IRC02:23
*** Sukhdev has joined #openstack-infra02:23
*** yamamoto has joined #openstack-infra02:23
*** ihrachys has quit IRC02:24
*** baoli has joined #openstack-infra02:27
*** yamamoto has quit IRC02:28
*** MarkAtwood has joined #openstack-infra02:28
*** MarkAtwood has quit IRC02:29
*** jpr1 has quit IRC02:32
*** baoli has quit IRC02:33
*** sridhar_ram has joined #openstack-infra02:34
*** tonytan4ever has quit IRC02:37
*** boris-42 has joined #openstack-infra02:38
*** MarkAtwood has joined #openstack-infra02:38
*** salv-orlando has joined #openstack-infra02:42
openstackgerritKyle Mestery proposed openstack-infra/project-config: Enable requirements and make it voting for Kuryr  https://review.openstack.org/27444702:50
*** fedexo has joined #openstack-infra02:56
*** tiswanso has joined #openstack-infra02:57
*** shashank_hegde has joined #openstack-infra02:57
*** tiswanso has quit IRC03:01
*** tiswanso has joined #openstack-infra03:01
*** coolsvap|away is now known as coolsvap03:02
*** thorst has joined #openstack-infra03:21
*** yuanying has quit IRC03:22
*** yuanying has joined #openstack-infra03:22
*** Sukhdev has quit IRC03:22
*** Sukhdev has joined #openstack-infra03:23
*** salv-orlando has quit IRC03:26
*** thorst has quit IRC03:27
*** yamamoto_ has joined #openstack-infra03:33
*** gongysh_ has joined #openstack-infra03:38
*** links has joined #openstack-infra03:46
*** markvoelker has joined #openstack-infra03:47
*** mriedem_away has quit IRC03:50
*** spags has joined #openstack-infra03:50
*** markvoelker has quit IRC03:52
*** MarkAtwood has quit IRC03:54
*** Keedya_ has joined #openstack-infra03:57
*** flwang1 has quit IRC03:58
*** spags has quit IRC04:00
*** Keedya_ has quit IRC04:00
*** unicell has quit IRC04:02
*** yuanying has quit IRC04:06
*** unicell has joined #openstack-infra04:10
*** sridhar_ram has quit IRC04:15
*** tiswanso has quit IRC04:18
*** links has quit IRC04:19
*** keedya has quit IRC04:19
*** salv-orlando has joined #openstack-infra04:24
*** Sukhdev_ has joined #openstack-infra04:25
*** thorst has joined #openstack-infra04:26
*** julim_ has quit IRC04:26
*** shashank_hegde has quit IRC04:27
*** Sukhdev has quit IRC04:27
*** Sukhdev_ has quit IRC04:29
*** Sukhdev has joined #openstack-infra04:30
openstackgerritMatthew Thode proposed openstack/diskimage-builder: add new cloud-init element  https://review.openstack.org/27376404:33
prometheanfireSamYaple: thanks :D04:33
*** thorst has quit IRC04:33
prometheanfireSam-I-Am: didn't know you were in here04:33
*** yonglihe has quit IRC04:42
*** achanda has quit IRC04:44
*** Sukhdev has quit IRC04:44
*** baoli has joined #openstack-infra04:45
*** Sukhdev has joined #openstack-infra04:48
*** tphummel has joined #openstack-infra04:50
*** baoli has quit IRC04:50
*** zz_dimtruck is now known as dimtruck04:51
*** links has joined #openstack-infra04:53
*** achanda has joined #openstack-infra04:55
*** amotoki has joined #openstack-infra04:58
*** amotoki has quit IRC04:58
Sam-I-Amprometheanfire: yarp04:59
prometheanfireSam-I-Am: go to bed05:03
*** [1]Thelo has joined #openstack-infra05:04
*** chlong has quit IRC05:05
Sam-I-Amprometheanfire: yarp05:05
greghaynesprometheanfire: I reviewed all your dib patches05:05
greghaynesI think05:06
*** salv-orlando has quit IRC05:06
*** Thelo has quit IRC05:07
prometheanfiregreghaynes: hehe, I replied to your -105:07
*** [1]Thelo is now known as Thelo05:07
prometheanfirehttps://review.openstack.org/#/c/27379005:07
prometheanfiregreghaynes: I don't think anything should add itself to init on install05:07
greghaynesprometheanfire: oh, well install is conflated here. Glean is 'installed' via pip install glean05:08
greghaynesthat script is a helper script in glean rather than part of the 'install' for that reason05:08
prometheanfiregreghaynes: ok, well, it can go in there then, I still don't like adding init things to packages as much as possible05:09
*** amotoki has joined #openstack-infra05:10
greghaynesYea, it is wierd because of the python pacakging aspect - for a normal package we would just let the distro pacakge take care of it05:10
greghaynesfor a non distro pacakged python thing either dib needs to carry the code or the package needs to provide it, both if which are not ideal05:10
prometheanfireI see dib taking on the role of a distro here05:11
prometheanfiregreghaynes: mind if I keep it with [[]]05:11
greghaynesprometheanfire: eh, I think that will be the only place we use that for the debug conditional so it breaks our consistency...05:12
prometheanfiregreghaynes: I've been told back and forth on that05:12
greghaynesya, whoever told you otherwise is wrong ;)05:12
prometheanfireof course05:12
prometheanfireand they'd say the same to you05:12
greghaynesthe rule is [] unless you need something in the builtin05:12
greghaynesWe decided this as a project a while ago...05:12
prometheanfirebad decision imo, but ok05:13
greghaynesMy opinion on it was pick one and do it05:13
prometheanfireI really don't want to have to go back and change all the [[ I've done to [05:13
greghaynesso *shrug*05:13
openstackgerritMatthew Thode proposed openstack/diskimage-builder: add new cloud-init element  https://review.openstack.org/27376405:13
greghaynesYea, sorry about that :(05:13
greghaynesRe simple-init init, I think I agree with you05:14
greghaynesbut05:14
prometheanfireI'm not doing it yet, just to the DIB_DEBUG_TRACE I've changed/added05:14
greghayneswe should still add it to glean for now so we match the other init systems05:15
greghaynesif we want to do init config in dib then we should move all of them in to there05:15
greghaynesThat is also the kind of thing which would be good to discuss with other glean cores before actually doing05:16
prometheanfireya, I get consistency05:16
prometheanfireanyway, I'm caught up, except for simple-init05:16
*** salv-orlando has joined #openstack-infra05:17
*** chlong has joined #openstack-infra05:17
*** rguillebert has quit IRC05:17
*** ramishr__ has joined #openstack-infra05:18
greghaynesprometheanfire: on that cloud-init element, do you not need a package map?05:19
prometheanfireit's called cloud-init in gentoo05:19
greghaynesah, ok05:19
greghaynesI am not entirely clear on when you need to foo/package vs no05:20
prometheanfireit'd be more correct to have a pkg-map for cloud-init to map to app-emulation/cloud-init, but both work05:20
greghaynesok05:20
prometheanfireit's needed for git because there are two 'git's in the tree05:20
prometheanfiredev-ruby/git and dev-vcs/git05:20
greghayneseek, that seems scary from a backwards compat standpoint05:20
prometheanfireother times it's a package name change05:20
prometheanfirebackwards compat?05:21
greghaynessomeone adds another package with the same name and all of a sudden everyone who doesnt prefix the package breaks?05:21
prometheanfireon install it's added the the 'world' file as the full package name05:21
prometheanfireif it's just being installed as a dep our ebuilds also use the full name05:21
prometheanfireno issue05:21
greghaynesI just mean for something like dib as an example, if someone added foo/cloud-init we would break05:21
openstackgerritMatthew Thode proposed openstack-infra/glean: enable glean in gentoo's init on install  https://review.openstack.org/27450505:21
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system  https://review.openstack.org/27377205:21
prometheanfireheh05:22
greghaynesI am not really worried about that specific case, but ya05:22
openstackgerritMatthew Thode proposed openstack/diskimage-builder: add support for gentoo to simple-init  https://review.openstack.org/27379005:23
*** chlong has quit IRC05:25
prometheanfiregreghaynes: how's this for a pkg-map? https://bpaste.net/show/8aba9005e5bc05:25
prometheanfiredon't think I need to define a default05:26
greghaynesprometheanfire: Thats fine, I am actually +2 on the patch, I was mostly just commenting on the system as a whole05:26
*** ramishr__ has quit IRC05:26
greghaynesprometheanfire: I am going to update your commit msg on https://review.openstack.org/#/c/273790/3 to not mention enabling the glean service05:26
prometheanfireI'm still gonna add it05:26
prometheanfiresure05:27
*** ramishr__ has joined #openstack-infra05:27
openstackgerritMatthew Thode proposed openstack/diskimage-builder: add new cloud-init element  https://review.openstack.org/27376405:27
prometheanfirethough I may get there first05:27
openstackgerritgreghaynes proposed openstack/diskimage-builder: add support for gentoo to simple-init  https://review.openstack.org/27379005:28
prometheanfireoh boo05:28
greghaynesI just use the gerrit web ui for that all the time05:28
prometheanfireI would have been faster if I didn't work on the cloud-init commit :P05:29
*** zhenguo_ has quit IRC05:30
*** ramishr__ has quit IRC05:30
prometheanfiregreghaynes: https://review.openstack.org/#/c/273772/ https://review.openstack.org/#/c/273795/ https://review.openstack.org/#/c/274505/05:31
prometheanfiregreghaynes: think those are it05:31
greghaynesprometheanfire: ah, ok05:31
*** thorst has joined #openstack-infra05:31
prometheanfireyep, confirmed05:32
greghayneshrm, gerrit is not happ6y05:32
*** ramishr__ has joined #openstack-infra05:32
prometheanfireya, had to refresh05:33
*** amotoki has quit IRC05:33
*** amotoki has joined #openstack-infra05:34
greghaynesprometheanfire: You might also consider making a patch to add gentoo to http://docs.openstack.org/developer/diskimage-builder/user_guide/supported_distros.html05:35
*** salv-orlando has quit IRC05:35
prometheanfiregreghaynes: after this, sure05:35
prometheanfirenot need these 7 reviews :P05:36
prometheanfirejust05:36
*** gongysh_ has quit IRC05:36
prometheanfireadded to my todo for next sunday, hopefully my patches will be in by then05:37
greghaynescool05:37
*** thorst has quit IRC05:37
*** chlong has joined #openstack-infra05:38
*** coolsvap is now known as coolsvap|away05:40
*** denisra_ has joined #openstack-infra05:40
*** Sukhdev has quit IRC05:41
greghaynesclarkb: When youre awake - ianw is looking for input on https://review.openstack.org/#/c/268241/05:41
*** mtanino___ has joined #openstack-infra05:42
*** ikalnitsky_ has joined #openstack-infra05:42
*** kuromagi^ has joined #openstack-infra05:43
*** trown has joined #openstack-infra05:43
*** DuncanT_ has joined #openstack-infra05:43
*** darrenc_ has joined #openstack-infra05:43
*** wendar_ has joined #openstack-infra05:43
*** fedexo_ has joined #openstack-infra05:43
*** Nakato_ has joined #openstack-infra05:44
*** matt6434 has joined #openstack-infra05:44
*** pfalleno1 has joined #openstack-infra05:45
*** gus_ has joined #openstack-infra05:45
*** bradm_ has joined #openstack-infra05:45
*** jokke__ has joined #openstack-infra05:45
*** gibi_ has joined #openstack-infra05:45
*** Anticime1 has joined #openstack-infra05:45
*** lifeless_ has joined #openstack-infra05:45
*** fabo_ has joined #openstack-infra05:45
*** ashleighfarnham has joined #openstack-infra05:45
*** _fortis_ has joined #openstack-infra05:46
openstackgerritMerged openstack/diskimage-builder: Remove zero length files  https://review.openstack.org/27055005:46
*** amotoki has quit IRC05:47
*** achanda_ has joined #openstack-infra05:47
prometheanfiregreghaynes: dib workflow is broken atm I think05:47
greghaynesyou mean merging?05:47
*** Sukhdev has joined #openstack-infra05:47
prometheanfireso https://review.openstack.org/#/c/273769/ won't work05:48
*** anthonyper_ has joined #openstack-infra05:48
*** onovy_ has joined #openstack-infra05:48
*** jmccrory_ has joined #openstack-infra05:48
greghaynesprometheanfire: yep, but you can recheck whenever the gate is un-broke and itll merge05:48
prometheanfirecool05:48
*** markvoelker has joined #openstack-infra05:48
*** dimtruck_ has joined #openstack-infra05:48
*** med_ has joined #openstack-infra05:48
*** med_ is now known as Guest8071105:49
*** fedexo has quit IRC05:49
*** changzhi has quit IRC05:49
*** Anticimex has quit IRC05:49
*** DuncanT has quit IRC05:49
*** _fortis has quit IRC05:49
*** pfallenop has quit IRC05:49
*** denisra has quit IRC05:49
*** lifeless has quit IRC05:49
*** madorn has quit IRC05:49
*** krtaylor has quit IRC05:49
*** mgagne has quit IRC05:49
*** mtanino__ has quit IRC05:49
*** aarefiev has quit IRC05:49
*** onovy has quit IRC05:49
*** bswartz has quit IRC05:49
*** jmccrory has quit IRC05:49
*** dtroyer has quit IRC05:49
*** luv has quit IRC05:49
*** Guest65103 has quit IRC05:49
*** jokke_ has quit IRC05:49
*** Kennan has quit IRC05:49
*** _diana_ has quit IRC05:49
*** lewo has quit IRC05:49
*** comstud has quit IRC05:49
*** gibi has quit IRC05:49
*** ikalnitsky has quit IRC05:49
*** trown|outttypeww has quit IRC05:49
*** bradm has quit IRC05:49
*** Zara_ has quit IRC05:49
*** mattoliverau has quit IRC05:49
*** dimtruck has quit IRC05:49
*** gus has quit IRC05:49
*** jamielennox has quit IRC05:49
*** Nakato has quit IRC05:49
*** kuromagi has quit IRC05:49
*** anthonyper has quit IRC05:49
*** darrenc has quit IRC05:49
*** fabo has quit IRC05:49
*** isviridov_away has quit IRC05:49
*** vaidy has quit IRC05:49
*** d34dh0r53 has quit IRC05:49
*** dolphm has quit IRC05:49
*** eglute has quit IRC05:49
*** cloudnull has quit IRC05:49
*** sigmavirus24_awa has quit IRC05:49
*** wendar has quit IRC05:49
*** fungi has quit IRC05:49
*** anthonyper_ is now known as anthonyper05:49
*** dimtruck_ is now known as dimtruck05:49
*** onovy_ is now known as onovy05:49
*** jmccrory_ is now known as jmccrory05:49
*** achanda has quit IRC05:49
*** mgagne_ has joined #openstack-infra05:49
*** coolsvap|away is now known as coolsvap05:49
*** mtanino___ is now known as mtanino__05:49
*** luv has joined #openstack-infra05:49
*** dolphm has joined #openstack-infra05:50
*** ikalnitsky_ is now known as ikalnitsky05:50
*** dtroyer has joined #openstack-infra05:50
*** ashleighfarnham has quit IRC05:50
*** DuncanT_ is now known as DuncanT05:51
*** lewo has joined #openstack-infra05:51
*** d34dh0r53 has joined #openstack-infra05:52
*** comstud has joined #openstack-infra05:52
*** markvoelker has quit IRC05:53
*** sigmavirus24_awa has joined #openstack-infra05:54
*** amotoki has joined #openstack-infra05:54
*** eglute has joined #openstack-infra05:55
*** MarkAtwood has joined #openstack-infra05:55
*** krtaylor has joined #openstack-infra05:56
*** aarefiev has joined #openstack-infra05:56
*** _diana_ has joined #openstack-infra05:56
*** madorn has joined #openstack-infra05:57
*** fungi has joined #openstack-infra05:57
*** Kennan has joined #openstack-infra05:58
*** ramishr__ is now known as ramishra05:59
openstackgerritMatthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system  https://review.openstack.org/27377206:01
openstackgerritMatthew Thode proposed openstack/diskimage-builder: fix gentoo hardened support  https://review.openstack.org/27379506:01
prometheanfiregreghaynes: mind adding +W to https://review.openstack.org/#/c/273776/06:02
greghaynesprometheanfire: done06:03
prometheanfirethanks06:03
*** amotoki has quit IRC06:05
*** jamielennox|away has joined #openstack-infra06:07
*** jamielennox|away is now known as jamielennox06:07
*** abregman has joined #openstack-infra06:07
*** _fortis_ is now known as _fortis06:07
*** abregman has quit IRC06:08
*** abregman has joined #openstack-infra06:08
*** markmcclain has quit IRC06:13
*** salv-orlando has joined #openstack-infra06:13
*** amotoki has joined #openstack-infra06:14
*** markmcclain has joined #openstack-infra06:15
*** toabctl is now known as toabctl_06:17
*** hichihara has joined #openstack-infra06:19
jlvillalAnyone know how long it takes for a new release to propagate from PyPi to the gate system?  This update to upper-constraints.txt keeps failing: https://review.openstack.org/#/c/274462/1  I can download the 1.1.0 version of python-ironicclient from PyPi.06:19
jlvillalThe log file says: No matching distribution found for python-ironicclient===1.1.0 (from -c /opt/stack/new/requirements/upper-constraints.txt (line 286))06:21
jlvillalhttp://logs.openstack.org/62/274462/1/check/gate-tempest-dsvm-full/c6252fe/logs/devstacklog.txt.gz#_2016-02-01_03_04_12_65106:21
*** markmcclain has quit IRC06:21
*** isviridov_away has joined #openstack-infra06:23
*** MarkAtwood has quit IRC06:24
*** tphummel has quit IRC06:24
*** vaidy_ril has joined #openstack-infra06:26
*** nikil89_ has joined #openstack-infra06:26
*** nikil89_ is now known as naggappan06:26
*** aopincar has joined #openstack-infra06:29
jlvillalAs a note, until the above patch is merged the Ironic gate is broken.06:30
*** rossella_s has quit IRC06:31
*** rossella_s has joined #openstack-infra06:31
*** mrmartin has joined #openstack-infra06:33
*** thorst has joined #openstack-infra06:36
*** cloudnul- has joined #openstack-infra06:38
*** Nakato_ has quit IRC06:40
*** Daisy has joined #openstack-infra06:41
*** chlong has quit IRC06:41
*** thorst has quit IRC06:42
*** Nakato has joined #openstack-infra06:42
*** markmcclain has joined #openstack-infra06:43
*** _nadya_ has joined #openstack-infra06:46
*** chlong has joined #openstack-infra06:46
*** fabo_ is now known as fabo06:47
*** markmcclain has quit IRC06:48
*** hdd has quit IRC06:50
*** cloudnul- is now known as cloudnull06:51
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/27390706:52
*** salv-orlando has quit IRC06:57
*** achanda_ is now known as achanda06:57
*** esikachev has joined #openstack-infra06:59
*** gildub has quit IRC07:01
*** kushal has joined #openstack-infra07:02
*** [HeOS] has quit IRC07:05
*** [HeOS] has joined #openstack-infra07:05
*** salv-orlando has joined #openstack-infra07:09
DaisyStevenK: ping07:10
*** followinfrared has joined #openstack-infra07:13
*** zul has joined #openstack-infra07:14
*** sbalukoff has joined #openstack-infra07:15
*** markmcclain has joined #openstack-infra07:17
AJaegerjlvillal: we updated our mirroring setup, might be that something is not working as it should. jeblair, krotschek, could you check python-ironicclient, please? We need version 1.1.0...07:19
*** yamahata has joined #openstack-infra07:21
*** markmcclain has quit IRC07:22
*** hichihara has quit IRC07:24
*** jaosorior has joined #openstack-infra07:26
*** scheuran has joined #openstack-infra07:26
*** mrmartin has quit IRC07:26
*** jaosorior has quit IRC07:26
*** jaosorior has joined #openstack-infra07:27
*** jtomasek has joined #openstack-infra07:32
*** jyuso1 has joined #openstack-infra07:35
*** nmagnezi has joined #openstack-infra07:37
*** thorst has joined #openstack-infra07:40
*** jaosorior has quit IRC07:41
*** salv-orlando has quit IRC07:43
*** chlong has quit IRC07:46
*** placko has joined #openstack-infra07:46
*** thorst has quit IRC07:47
*** markvoelker has joined #openstack-infra07:49
*** shashank_hegde has joined #openstack-infra07:52
*** amotoki has quit IRC07:52
*** zul has quit IRC07:52
*** ttx has joined #openstack-infra07:52
*** markvoelker has quit IRC07:54
*** jokke__ is now known as jokke_07:56
*** shashank_hegde has quit IRC07:59
*** zeih has joined #openstack-infra08:00
*** dimtruck is now known as zz_dimtruck08:03
*** alivigni has joined #openstack-infra08:04
*** yamamoto_ has quit IRC08:05
*** salv-orlando has joined #openstack-infra08:06
*** mikelk has joined #openstack-infra08:08
*** yamamoto_ has joined #openstack-infra08:08
*** gildub has joined #openstack-infra08:10
*** tosky has joined #openstack-infra08:10
*** arxcruz has joined #openstack-infra08:12
*** ildikov has quit IRC08:12
*** yamamoto_ has quit IRC08:13
*** mkoderer has quit IRC08:14
*** dcentral has joined #openstack-infra08:14
*** dcentral has quit IRC08:19
yolandamorning08:19
*** dcentral has joined #openstack-infra08:20
zeihmorning08:21
AJaegergood morning, yolanda and zeih.08:21
AJaegeryolanda: did everything go fine yesterday?08:21
*** dcentral has quit IRC08:22
*** yamamoto_ has joined #openstack-infra08:23
nibalizermorning yall08:23
*** yamamoto_ has quit IRC08:24
yolandaAJaeger, still working on it08:24
*** amotoki has joined #openstack-infra08:24
AJaegermorning, nibalizer !08:25
*** salv-orlando has quit IRC08:25
AJaegeryolanda: I hope you had some sleep...08:25
yolandaAJaeger, yes, i could take some rest, we had some good progress yesterday but we still have to finish today08:26
*** rcernin has joined #openstack-infra08:26
*** yamamoto_ has joined #openstack-infra08:27
AJaegerI see - then have a successfull day ;)08:27
*** yamamoto_ has quit IRC08:28
yolandathanks, i hope so :)08:28
*** changzhi has joined #openstack-infra08:29
*** d0ugal_ has quit IRC08:30
*** d0ugal has joined #openstack-infra08:30
*** achanda has quit IRC08:31
*** camunoz has quit IRC08:31
nibalizerAJaeger: did you have some things you wanted reviews on?08:31
*** amotoki has quit IRC08:33
*** yamahata has quit IRC08:34
*** sshnaidm has quit IRC08:35
*** coolsvap is now known as coolsvap|away08:36
*** Sukhdev has quit IRC08:36
*** achanda has joined #openstack-infra08:37
*** dchen has joined #openstack-infra08:37
*** dchen has quit IRC08:38
*** amotoki has joined #openstack-infra08:38
*** achanda has quit IRC08:39
*** ifarkas has joined #openstack-infra08:39
*** shardy has joined #openstack-infra08:42
*** dingyichen has quit IRC08:42
openstackgerritLuigi Toscano proposed openstack-infra/project-config: Fix Sahara config for DevStack on stable branches  https://review.openstack.org/27454908:42
*** mkoderer has joined #openstack-infra08:43
toskyand that's ^ should properly fix stable/kilo for Sahara08:44
*** yamamoto_ has joined #openstack-infra08:44
*** samuelBartel has joined #openstack-infra08:45
*** yamamoto_ has quit IRC08:45
*** thorst has joined #openstack-infra08:46
*** fedexo_ has quit IRC08:46
*** zz_dimtruck is now known as dimtruck08:47
*** davidlenwell has quit IRC08:47
*** yamamoto_ has joined #openstack-infra08:48
*** jlanoux has joined #openstack-infra08:51
*** thorst has quit IRC08:52
*** yamamoto_ has quit IRC08:53
AJaegernibalizer: yes, thanks for asking08:53
AJaegernibalizer: https://review.openstack.org/274352 and https://review.openstack.org/274342 , please08:54
AJaegertosky: I'll review...08:54
toskyAJaeger: thanks!08:55
*** davidlenwell has joined #openstack-infra08:56
*** yamamoto_ has joined #openstack-infra08:56
*** dimtruck is now known as zz_dimtruck08:56
AJaegernibalizer: https://review.openstack.org/273032 and https://review.openstack.org/273010 would be nice as well08:56
nibalizerAJaeger: kk08:57
AJaegernibalizer: and other reviews: tosky's 274549, https://review.openstack.org/272469 https://review.openstack.org/27444708:57
*** fhubik has joined #openstack-infra08:59
toskyAJaeger: out of curiosity, are the jobs rebuilt immediately (or after a short time) when a change is merged to the jjb yamls?08:59
*** lezbar has joined #openstack-infra08:59
AJaegertosky: jobs are deployed with puppet, puppet basically runs every 30 mins.09:00
toskyah, ok09:00
AJaegertosky: So, my rule of thumb: an hour after a merge, it should be propagaded to all systems and it's safe to "recheck"09:00
toskyperfect :)09:00
*** mohankumar has joined #openstack-infra09:01
*** Hal has joined #openstack-infra09:03
*** yaume has joined #openstack-infra09:03
*** Hal is now known as Guest3781409:03
*** jyuso1 has quit IRC09:03
yolandanibalizer, i put a -1 on https://review.openstack.org/#/c/274252/1/templates/statusbot.config.erb . Is that a bug, or do i miss some dependent change where it changes the name of the parameter?09:04
*** yamamoto_ has quit IRC09:04
*** zhurong has quit IRC09:06
*** salv-orlando has joined #openstack-infra09:07
*** fhubik has quit IRC09:08
*** zul has joined #openstack-infra09:11
*** yamamoto_ has joined #openstack-infra09:12
jesusaurnibalizer: how was fossdem?09:12
*** yamamoto_ has quit IRC09:13
*** jordanP has joined #openstack-infra09:13
*** zhenguo_ has joined #openstack-infra09:16
*** ildikov has joined #openstack-infra09:18
*** fbo_ is now known as fbo09:18
nibalizerjesusaur: very nice09:20
nibalizernow at config mgmt camp09:20
*** amotoki has quit IRC09:20
*** jistr has joined #openstack-infra09:20
*** yamamoto_ has joined #openstack-infra09:21
nibalizeryolanda: huh i thought i updated this09:23
openstackgerritSpencer Krum proposed openstack-infra/puppet-statusbot: Allow statusbot to take a hash of auth nicks  https://review.openstack.org/27425209:23
nibalizerupdated :)09:23
*** zul has quit IRC09:25
*** salv-orlando has quit IRC09:27
*** markmcclain has joined #openstack-infra09:31
*** sshnaidm has joined #openstack-infra09:31
*** zul has joined #openstack-infra09:32
openstackgerritMerged openstack-infra/project-config: Add openstack-salt project and jobs  https://review.openstack.org/27246909:32
*** fhubik has joined #openstack-infra09:35
*** ociuhandu has quit IRC09:37
*** ajo has joined #openstack-infra09:37
openstackgerritMerged openstack-infra/project-config: Move glanceclient-dsvm-function to glanceclient  https://review.openstack.org/27303209:38
*** markmcclain has quit IRC09:40
*** fhubik is now known as fhubik_brb09:41
*** fhubik_brb is now known as fhubik09:41
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove jobs from gantt and python-ganttclient  https://review.openstack.org/27301009:42
*** ajo has quit IRC09:42
*** ajo has joined #openstack-infra09:43
*** acruz has joined #openstack-infra09:43
*** electrofelix has joined #openstack-infra09:44
*** matrohon has joined #openstack-infra09:45
*** arxcruz has quit IRC09:45
*** ashleighfarnham has joined #openstack-infra09:47
*** ajo has quit IRC09:48
*** ajo has joined #openstack-infra09:48
*** ajo has quit IRC09:49
*** markvoelker has joined #openstack-infra09:50
*** thorst has joined #openstack-infra09:50
*** zul has quit IRC09:51
*** ashleighfarnham has quit IRC09:52
*** ramishra has quit IRC09:52
*** ramishra_ is now known as ramishra09:53
*** markvoelker has quit IRC09:54
*** pblaho has joined #openstack-infra09:56
*** amotoki has joined #openstack-infra09:56
*** thorst has quit IRC09:57
*** BobBall has quit IRC09:58
*** BobBall has joined #openstack-infra09:59
*** vgridnev has joined #openstack-infra10:02
*** openstackgerrit has quit IRC10:02
*** openstackgerrit_ has joined #openstack-infra10:02
*** openstackgerrit_ has quit IRC10:03
*** salv-orlando has joined #openstack-infra10:03
*** zeih has quit IRC10:05
*** sc68cal__ has joined #openstack-infra10:07
*** yamamoto_ has quit IRC10:08
*** matrohon has quit IRC10:08
*** achanda has joined #openstack-infra10:09
*** matrohon has joined #openstack-infra10:10
*** dtardivel has joined #openstack-infra10:10
*** notnownikki has joined #openstack-infra10:13
*** achanda has quit IRC10:14
*** sc68cal__ is now known as sc68cal10:17
*** sc68cal is now known as Guest6916410:18
*** Guest69164 has quit IRC10:19
*** Guest69164 has joined #openstack-infra10:19
*** mrmartin has joined #openstack-infra10:20
*** ttx_ has joined #openstack-infra10:21
*** mkoderer__ has joined #openstack-infra10:21
*** ttx has quit IRC10:23
mescanefhello, please review https://review.openstack.org/#/c/269007/ - only WF+1 needed:-)10:24
*** ttx_ is now known as ttx10:24
*** daemontool_ has joined #openstack-infra10:25
*** daemontool has quit IRC10:28
*** ldnunes has joined #openstack-infra10:29
*** rossella_s has quit IRC10:31
*** rossella_s has joined #openstack-infra10:32
*** jokke_ has quit IRC10:34
*** jokke_ has joined #openstack-infra10:34
*** marekd has quit IRC10:34
*** gildub has quit IRC10:36
mrmartinmorning10:39
*** salv-orlando has quit IRC10:40
*** marekd has joined #openstack-infra10:42
*** christiana has quit IRC10:45
*** zul has joined #openstack-infra10:46
EmilienMAJaeger: hey, when you got tome, can you look https://review.openstack.org/#/c/274137/ please ? I don't want to break something :-)10:46
*** zz_dimtruck is now known as dimtruck10:47
*** deepakcs has joined #openstack-infra10:49
deepakcsHi infra cores, can someone take a look and bless : https://review.openstack.org/#/c/273326/10:50
*** salv-orlando has joined #openstack-infra10:50
deepakcsjohnthetubaguy: when u have time: https://review.openstack.org/#/c/273326/10:50
*** rfolco has joined #openstack-infra10:50
AJaegerEmilienM: I commented already, did you see my note?10:52
EmilienMAJaeger: if you don't -1, I don't see it, sorry for that10:52
EmilienMAJaeger: don't hesitate to -1 me :-P10:53
* EmilienM is not reading Gerrit e-mails, shamelessly :-)10:53
AJaegerEmilienM: Ok, will give -1 instead of 0 and -2 instead of -1. Correct?10:53
EmilienMlool10:53
EmilienMAJaeger++10:53
mrmartinhi guys10:54
mrmartindo we have an infra-core here?10:54
deepakcsAJaeger: when u have some time: https://review.openstack.org/#/c/273326/10:55
EmilienMAJaeger: I was wondering the layout ordering, but I'll let you review it, in case I missed something10:55
nibalizermrmartin: ish10:55
mrmartinhi nibalizer10:56
mrmartinso it is not sure, that the code is wrong, had you made the backup yesterday of ask.o.o db?10:56
AJaegerEmilienM: I replaced just the last entry and run a diff and it looked fine - but will double check once you submit it.10:56
*** thorst has joined #openstack-infra10:56
AJaegerdeepakcs: I'd like to see PTLs +1 first but let me comment on that...10:56
*** fhubik is now known as fhubik_brb10:56
mrmartinI want a move-infra-cores to EU travel program.10:57
mrmartin:)10:57
EmilienMI think openstack bot is dead10:57
*** ildikov has quit IRC10:57
EmilienMAJaeger: https://review.openstack.org/#/c/274137/1 updated - not urgent10:57
nibalizermrmartin: i didn't take a db backup yesterday no10:58
mrmartinnibalizer: can you do one, and copy it to ask-staging.o.o somewhere?10:58
deepakcsAJaeger: thanks10:59
mrmartinI'll try to restore there, and check whether the branch changes confused things there.10:59
deepakcsI pinged PTLs too, lets see :)10:59
mrmartinnibalizer: this is the etherpad https://etherpad.openstack.org/p/askbot-migration210:59
AJaegerEmilienM: will review later11:00
EmilienMno urgency for me, thanks11:00
mrmartinI started to write a better ops guide yesterday for ask.o.o but cannot finish due the turbulence on the plane.11:00
*** dimtruck is now known as zz_dimtruck11:01
AJaegerIs gerritbot dead? I don't see a notification here for EmilienM's update of 274137...11:02
EmilienMsame11:02
EmilienMEmilienM | I think openstack bot is dead11:02
nibalizerhrm11:02
*** thorst has quit IRC11:03
*** fhubik_brb is now known as fhubik11:03
AJaegernibalizer: did it crash after the import of I705c9f2384efcaf4398c6662acdf0cff4b6ed348 ? Or just not restart properly?11:03
nibalizerAJaeger: lookng11:05
*** dims has joined #openstack-infra11:09
*** ociuhandu has joined #openstack-infra11:10
*** achanda has joined #openstack-infra11:11
*** tosky has quit IRC11:12
*** aysyd has joined #openstack-infra11:12
*** erlon has joined #openstack-infra11:13
*** Guest69164 has quit IRC11:13
*** matt6434 is now known as mattoliverau11:14
*** yamamoto has joined #openstack-infra11:17
*** achanda has quit IRC11:17
*** lifeless_ is now known as lifeless11:17
*** gnuoy_ has joined #openstack-infra11:19
*** gnuoy_ has quit IRC11:19
*** changzhi has quit IRC11:21
*** fhubik is now known as fhubik_brb11:21
*** Daisy has quit IRC11:22
nibalizerAJaeger: so gerritbot in logs is humming along11:23
*** yamamoto has quit IRC11:23
*** yamamoto has joined #openstack-infra11:24
*** jpr has joined #openstack-infra11:26
*** yamamoto has quit IRC11:28
*** yamamoto has joined #openstack-infra11:31
*** yamahata has joined #openstack-infra11:32
nibalizerAJaeger: is it possible gerritbot just netsplit?11:35
*** yamamoto has quit IRC11:36
*** jaosorior has joined #openstack-infra11:36
deepakcsAJaeger: you mean i need to remove gate-tempest-dsvm-full-ceph from under devstack-jobs as well as the template itself, right ?11:40
*** aarefiev has quit IRC11:48
*** markvoelker has joined #openstack-infra11:50
*** mrmartin has quit IRC11:51
AJaegerdeepakcs: If it's not needed anymore: Yes, remove it everywhere11:51
deepakcsAJaeger: ok done, just 1 Q i see some jobs under devstack-jobs and some aren't, whats the signifgicance of adding a job under devstack-jobs11:52
deepakcsAJaeger: in jenkins/jobs/devstack-gate.yaml11:52
*** mohankumar_ has joined #openstack-infra11:52
*** mohankumar has quit IRC11:52
AJaegerdeepakcs: there's a lot of cargo cult and I don't have the history. If your job runs on several repos, you can add it but you don't need to...11:53
nibalizerAttributeError: 'GerritBot' object has no attribute 'reconnect'11:53
nibalizerinteresting11:53
*** openstackgerrit has joined #openstack-infra11:53
AJaegernibalizer: yes, that's missing. WE have to restart it, do our scripts not do it this way?11:53
*** jlanoux has quit IRC11:53
*** openstackgerrit has quit IRC11:54
*** openstackgerrit_ has joined #openstack-infra11:54
nibalizer:shrug: ?11:54
*** openstackgerrit_ is now known as openstackgerrit11:55
*** markvoelker has quit IRC11:55
*** aarefiev has joined #openstack-infra11:55
deepakcsAJaeger: ok, thanks, i sent another patch with ur issue addressed. https://review.openstack.org/#/c/273326/11:55
AJaegernibalizer: I'm only aware of http://docs.openstack.org/infra/system-config/irc.html#troubleshooting as help here...11:56
*** flepied has joined #openstack-infra11:56
nibalizerya11:57
*** flepied1 has quit IRC11:59
*** openstackgerrit has quit IRC11:59
*** esikachev has quit IRC12:00
*** thorst has joined #openstack-infra12:01
*** salv-orl_ has joined #openstack-infra12:03
*** baoli has joined #openstack-infra12:04
* AJaeger just noticed that the openstack-salt repository uses submodules, it has a .gitmodules file.12:04
AJaegerI remember some discussion about submodules years ago, is that setup valid, fungi?12:05
*** baoli has quit IRC12:05
*** yamahata has quit IRC12:06
*** salv-orlando has quit IRC12:06
nibalizerAJaeger: so i've restarted the bot a few times12:07
nibalizernuthin12:07
*** openstackgerrit has joined #openstack-infra12:07
nibalizer gerritbot: Joined channel #openstack-kiloeyes12:08
AJaeger;( Sorry, no idea. Thanks for trying, nibalizer12:08
*** thorst has quit IRC12:08
nibalizerI think its working now12:08
*** baoli has joined #openstack-infra12:08
*** baoli has quit IRC12:08
AJaegerYes, openstackgerrit is here...12:08
AJaegerGreat! Thanks for fixing ;)12:09
nibalizerheh12:09
nibalizeri'll restart something several times any day :P12:09
AJaeger;)12:10
nibalizermrmartin.... you left :(12:10
nibalizerI am working on your db12:10
*** exploreshaifali has joined #openstack-infra12:12
*** rguillebert has joined #openstack-infra12:13
*** esikachev has joined #openstack-infra12:15
*** gordc has joined #openstack-infra12:16
*** jaosorior has quit IRC12:21
openstackgerritMerged openstack-infra/project-config: Re-enable oslo translations  https://review.openstack.org/27434212:21
*** jcoufal has joined #openstack-infra12:21
*** jaosorior has joined #openstack-infra12:21
openstackgerritMerged openstack-infra/project-config: gate-mistral-docker-buildimage has been moved to post  https://review.openstack.org/27448612:22
*** esikachev has quit IRC12:23
openstackgerritMerged openstack-infra/project-config: Fix Sahara config for DevStack on stable branches  https://review.openstack.org/27454912:23
*** esikachev has joined #openstack-infra12:24
openstackgerritMerged openstack-infra/project-config: Enable requirements and make it voting for Kuryr  https://review.openstack.org/27444712:24
openstackgerritMerged openstack-infra/project-config: Make gate-solum-python34 voting; add gate job  https://review.openstack.org/27342512:25
*** christiana has joined #openstack-infra12:25
mordredAJaeger: yup - submodules are totally valid - and in fact, gerrit can do neat things with them12:26
openstackgerritDeepak C Shetty (deepakcs) proposed openstack-infra/project-config: Ceph CI: Move from hook based job to plugin based job  https://review.openstack.org/27332612:26
mordredAJaeger: we don't normally use them in openstack projects because intreacting with them as a dev can be confusing12:26
openstackgerritMerged openstack-infra/project-config: Add translation jobs and script for translation of monasca-ui  https://review.openstack.org/27336112:27
*** weshay has quit IRC12:32
*** fhubik_brb is now known as fhubik12:36
Zaramordred: btw, the subscriptions page patch I linked you for storyboard.openstack.org has been merged. it's still pretty rudimentary. https://storyboard.openstack.org/#!/dashboard/subscriptions12:39
*** weshay_xchat has joined #openstack-infra12:40
*** bswartz has joined #openstack-infra12:41
*** jlanoux has joined #openstack-infra12:41
*** tosky has joined #openstack-infra12:42
*** doug-fish has joined #openstack-infra12:42
*** jlanoux has quit IRC12:47
*** salv-orl_ has quit IRC12:47
*** thorst has joined #openstack-infra12:48
EmilienMnibalizer: hey, are you still interested by https://review.openstack.org/219879 ? I think _ody was also looking at doing something like this12:49
*** sshnaidm has quit IRC12:50
*** markvoelker has joined #openstack-infra12:51
*** zz_dimtruck is now known as dimtruck12:52
*** weshay_xchat is now known as weshay12:52
*** gnuoy_ has joined #openstack-infra12:53
openstackgerritAndrea Frittoli proposed openstack-infra/log_processor: Load auto foward field prefix from config  https://review.openstack.org/27464512:53
*** mrmartin has joined #openstack-infra12:53
*** hashar has joined #openstack-infra12:54
*** realitix has joined #openstack-infra12:55
*** markvoelker has quit IRC12:56
*** MarkAtwood has joined #openstack-infra12:56
*** jlanoux has joined #openstack-infra12:57
*** Anticime1 is now known as Anticimex12:58
*** jaosorior has quit IRC13:00
*** mrmartin has quit IRC13:00
*** gnuoy_ has quit IRC13:01
*** mrmartin has joined #openstack-infra13:01
*** dimtruck is now known as zz_dimtruck13:01
*** salv-orlando has joined #openstack-infra13:02
*** gnuoy_ has joined #openstack-infra13:03
*** zul has quit IRC13:03
*** zeih has joined #openstack-infra13:04
*** dims has quit IRC13:06
jrollhey y'all, are we using pypi mirrors in CI now?13:06
jrollgot a fun one: http://logs.openstack.org/62/274462/1/check/gate-tempest-dsvm-full/43c74f5/logs/devstacklog.txt.gz#_2016-02-01_12_12_09_60613:06
*** shardy has quit IRC13:06
jrollit does exist on pypi https://pypi.python.org/pypi/python-ironicclient/1.1.013:07
*** gnuoy_ has quit IRC13:07
nibalizerEmilienM: ya i saw odys work13:09
nibalizeri thought his stuff was superior to what I was working with13:09
*** hashar has quit IRC13:09
*** dims has joined #openstack-infra13:09
*** hashar has joined #openstack-infra13:10
*** krotscheck1 has joined #openstack-infra13:11
*** krotscheck1 is now known as krotscheck_nobou13:11
*** xek__ is now known as xek13:12
*** naggappan has quit IRC13:13
*** markvoelker has joined #openstack-infra13:14
odyssey4meis it possible to have a tox env execute other tox envs? ie if I setup a 'linters' tox target, can I make that env simply execute the other already setup targets?13:15
*** edmondsw has joined #openstack-infra13:15
*** zul has joined #openstack-infra13:16
odyssey4meAJaeger ^ perhaps you know if that's possible - this is in relation to https://review.openstack.org/272723 which works, but could be simpler if a different approach was possible13:16
*** wei-wang has joined #openstack-infra13:17
*** flwang1 has joined #openstack-infra13:17
*** mattt has joined #openstack-infra13:18
*** fhubik is now known as fhubik_brb13:19
*** exploreshaifali has quit IRC13:20
*** fhubik_brb is now known as fhubik13:21
*** fhubik has quit IRC13:21
*** ildikov has joined #openstack-infra13:23
*** Daisy has joined #openstack-infra13:23
openstackgerritMiles Gould proposed openstack/requirements: python-ironicclient 1.1.0  https://review.openstack.org/27446213:23
*** Jeffrey4l_ has quit IRC13:23
openstackgerritSam Betts proposed openstack-infra/project-config: Add a tinyIPA non-voting test to Ironic  https://review.openstack.org/26968413:23
krotscheck_nobouMirros: Looks like gra1, iad, ord, hp region b, and sjc are entirely out of rotation. The others appear to have some leftover instances still using them.13:26
zeihhi, can someone have a look on https://review.openstack.org/#/c/271983/ ? in zuul all tests passed but in there is no +2 from jenkins :(13:26
*** baoli has joined #openstack-infra13:27
esikachevhi! i am need requirements core for merge https://review.openstack.org/#/c/270676/13:27
*** diana_clarke has joined #openstack-infra13:29
*** piet has joined #openstack-infra13:30
openstackgerritMerged openstack/requirements: Enforce compatibility job for tacker-horizon  https://review.openstack.org/27198313:32
AJaegerodyssey4me: there's macro substitution and you can reference another one but what you want does not work AFAIU13:32
AJaegerthanks mordred13:33
krotscheck_nobouzeih: That patch appears to be merged?13:33
zeihnow it has merged :)13:33
odyssey4methanks AJaeger - so the approach I'm taking is the only way to make it work then?13:33
AJaegerodyssey4me: let me check later a bit more and then comment on the review13:33
AJaegernibalizer, yolanda : Could either of you +2A this change for translations, please? https://review.openstack.org/#/c/274352/13:34
*** amrith is now known as _amrith_13:35
realitixHello everyone, can someone helps me with my registration. I followed all the procedures. I have a launchpad account, gerrit account, openstack foundation, all with the same name, id, and email. But when I try a "git review". I obtain "ICLA contributor agreement requires current contact information". But already did it.13:37
*** wznoinsk has joined #openstack-infra13:38
wznoinskhi infra13:38
wznoinskcould you have a quick look at https://review.openstack.org/#/c/269225/, could there be a problem with SCP when jenkins is asked to recheck?13:39
wznoinsksubsequest patchset 6 rechecks failed, a check on patchset 7 worked13:39
*** kzaitsev_mb has joined #openstack-infra13:40
*** sshnaidm has joined #openstack-infra13:41
openstackgerritBeth Elwell proposed openstack-infra/storyboard: Amended documentation naming of installations  https://review.openstack.org/27466113:43
andreafhi - I'd like to resume work on this log processor change https://review.openstack.org/#/c/222369/4 - but I'm not sure whether to continue here or on the log-processor repo instead (or both?)13:44
*** [HeOS] is now known as HeOS13:45
AJaegeryolanda: http://logs.openstack.org/25/269225/6/check/gate-intel-nfv-ci-tests-python27/825458e/console.html is again a "failed to upload" to /srv/static - is that disk causing problems again? Could you check, please?13:45
AJaegeryolanda: that's wznoinsk's comment from above13:46
yolandahi AJaeger, i can take a look13:46
yolandaa bit more free now13:46
*** yamamoto has joined #openstack-infra13:46
AJaegerthanks, yolanda - and thanks for the reviews!13:46
*** bauzas has quit IRC13:47
wznoinskAJaeger, thanks, it looks like scp sees the parent directory(ies) and doesn't need to create it but something is happning with creating the deeper dictory (no parent directory actually?)13:47
*** rlandy has joined #openstack-infra13:47
*** Jeffrey4l has joined #openstack-infra13:47
wznoinskyolanda,  AJaeger is there a launchpad space for these bugs?13:47
openstackgerritIllia Khudoshyn proposed openstack-infra/project-config: Add gate job for Rally against Keystone v2  https://review.openstack.org/27466813:48
*** diana_clarke has quit IRC13:48
*** ashleighfarnham has joined #openstack-infra13:48
mrmartinhi yolanda13:49
openstackgerritMerged openstack-infra/project-config: Do not sync translations: django_openstack_auth/designate-dashbaord  https://review.openstack.org/27435213:49
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Use new translation setup for all python projects  https://review.openstack.org/27442113:50
AJaegerwznoinsk: we use storyboard.openstack.org13:50
yolandawzoinsk, AJaeger, i don't see disk errors on static today13:50
*** ikhudoshyn has joined #openstack-infra13:51
*** pradk has joined #openstack-infra13:52
*** pradk has quit IRC13:52
*** ashleighfarnham has quit IRC13:53
*** cbader has quit IRC13:54
*** jaypipes has joined #openstack-infra13:54
*** yamamoto has quit IRC13:55
wznoinskyolanda, can we turn on some 'debuggin' mode from inside the tox.ini for scp/jenkins?13:57
*** yamamoto has joined #openstack-infra14:03
*** bauzas has joined #openstack-infra14:03
*** jaypipes has quit IRC14:03
*** yamamoto has quit IRC14:05
*** ayoung has joined #openstack-infra14:06
*** yamamoto has joined #openstack-infra14:08
*** amitgandhinz has joined #openstack-infra14:09
*** cbader has joined #openstack-infra14:10
openstackgerritMerged openstack/requirements: Bump upper constraint of futurist to 0.10.0  https://review.openstack.org/27324614:10
*** zz_dimtruck is now known as dimtruck14:12
*** jpr has quit IRC14:13
*** zul has quit IRC14:13
openstackgerritSam Betts proposed openstack-infra/project-config: Add a tinyIPA non-voting test to Ironic  https://review.openstack.org/26968414:13
*** julim has joined #openstack-infra14:14
EmilienMAJaeger: replied14:14
*** nelsnelson has joined #openstack-infra14:16
AJaegerEmilienM: My highlight was that it includes syntax tests as well - but your commit message has fewer tests AFAIU14:16
EmilienMAJaeger: oh... ok it's okay for me, we'll have lint tests in all the cases, so we're good14:17
*** nelsnels_ has quit IRC14:17
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Remove legacy pypi_trusted_hosts  https://review.openstack.org/27093014:17
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Removed pypi.<region> mirror manifests  https://review.openstack.org/27351814:17
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351914:17
*** fawadkhaliq has joined #openstack-infra14:17
*** sabeen1 has joined #openstack-infra14:18
*** fawadkhaliq has quit IRC14:18
AJaegerEmilienM: could you update the commit message, please?14:18
EmilienMAJaeger: of course14:18
AJaegerkrotscheck_nobou: do you have any ideas why python-ironicclient 1.1.0 is not mirrored?14:18
AJaegerkrotscheck_nobou: see failures of https://review.openstack.org/#/c/274462/14:19
openstackgerritEmilien Macchi proposed openstack-infra/project-config: zuul/layout: optimize puppet jobs  https://review.openstack.org/27413714:19
*** krotscheck_nobou has quit IRC14:19
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added afs to wheel mirror slaves  https://review.openstack.org/27305414:19
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Add AFS and Admin keytab to release slave.  https://review.openstack.org/27353814:19
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added AFS keytab to wheel slave  https://review.openstack.org/27320614:19
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added conditional install statements for kernel header packages  https://review.openstack.org/27320514:19
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added wheel hosting for AFS mirror slave  https://review.openstack.org/27097914:19
*** krotscheck1 has joined #openstack-infra14:19
* krotscheck1 peers at the bot14:20
*** fawadkhaliq has joined #openstack-infra14:20
*** nelsnelson has quit IRC14:20
AJaegerkrotscheck1: python-ironicclient 1.1.0 was released yesterday...14:20
*** nelsnelson has joined #openstack-infra14:21
DaisyHi, EmilienM. Can I get your support to https://review.openstack.org/#/c/274090/ ? It's to create a puppet module under openstack-infra to support the translation check website.14:21
AJaegerDaisy: you need only fungi's +1 and have it. This is a pure formality now...14:22
*** wendar_ is now known as wendar14:22
*** peristeri has joined #openstack-infra14:22
*** tiswanso has joined #openstack-infra14:22
krotscheck1AJaeger: ....and?14:22
*** sabeen3 has joined #openstack-infra14:22
*** mohankumar_ has quit IRC14:22
AJaegersee failures of https://review.openstack.org/#/c/274462/ - it cannot find the new release14:22
*** mohankumar_ has joined #openstack-infra14:22
krotscheck1AJaeger: Got it.14:22
krotscheck1Checking14:22
krotscheck1AJaeger: I'm guessing that bandersnatch is stuck.14:23
AJaegerDaisy: I commented, ttx mixed up the ownership.14:24
krotscheck1AJaeger: That's not something I have the ACL's, or experience, to fix.14:24
DaisyAJaeger: ok. Thank you.14:24
AJaegerttx, https://review.openstack.org/#/c/274090/414:24
AJaegerkrotscheck1: ok, we have to wait for jeblair then I guess14:24
DaisySorry, EmilienM.14:24
*** sabeen1 has quit IRC14:25
*** dguitarbite has quit IRC14:25
*** dguitarbite has joined #openstack-infra14:26
krotscheck1AJaeger: You might be able to get an initial diagnosis on mirror-update.o.o by looking at the log.14:26
EmilienMDaisy: no problem!14:26
EmilienMI'll +1 for fun14:27
AJaegerthanks, EmilienM !14:27
*** dtantsur|afk is now known as dtantsur14:27
*** krotscheck1 has quit IRC14:28
openstackgerritAndrea Frittoli proposed openstack-infra/puppet-log_processor: Create auto forwardable parameters  https://review.openstack.org/22236914:29
*** krotscheck1 has joined #openstack-infra14:29
*** eharney has joined #openstack-infra14:29
*** pabelanger has quit IRC14:30
*** pabelanger has joined #openstack-infra14:30
*** tiswanso has quit IRC14:31
*** jsavak has joined #openstack-infra14:31
*** rossella_s has quit IRC14:31
*** tiswanso has joined #openstack-infra14:31
*** rossella_s has joined #openstack-infra14:32
*** Daisy has quit IRC14:32
*** diana_clarke has joined #openstack-infra14:32
*** esikachev has quit IRC14:33
* krotscheck1 has been unable to find that log in logstash14:34
openstackgerritAndrea Frittoli proposed openstack-infra/puppet-log_processor: Create auto forwardable parameters  https://review.openstack.org/22236914:37
*** mrmartin has quit IRC14:39
krotscheck1AJaeger: If I call zuul-clone: project: openstack/requirements, what directory will that end up in? `pwd`/openstack/requirements?14:42
AJaegerodyssey4me: you can do something like "commands = {[testenv:pep8]commands}" - but it only works if {posargs} is not used. You should be able to add several of these substitution. Please try it out locally...14:42
AJaegerkrotscheck1: yes, AFAIK14:42
odyssey4meAJaeger :) thanks, let me try that out14:43
krotscheck1AJaeger: Gotcha, thnks14:43
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add IRC notification for windmill  https://review.openstack.org/27469414:45
*** zul has joined #openstack-infra14:45
*** deepakcs has quit IRC14:46
*** daemontool has joined #openstack-infra14:47
*** daemontool_ has quit IRC14:47
ttxAJaeger: +214:49
AJaegerthanks, ttx14:50
*** dimtruck is now known as zz_dimtruck14:50
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add missing node for windmill  https://review.openstack.org/27469714:51
*** piet has quit IRC14:52
*** daemontool has quit IRC14:53
*** aopincar has quit IRC14:53
*** yamamoto has quit IRC14:54
jrollfun one: http://logs.openstack.org/62/274462/1/check/gate-tempest-dsvm-full/43c74f5/logs/devstacklog.txt.gz#_2016-02-01_12_12_09_606 it does exist on pypi https://pypi.python.org/pypi/python-ironicclient/1.1.014:54
jrollI suspect pypi mirrors in use and not syncing right?14:54
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite  https://review.openstack.org/27470114:55
*** krotscheck1 has quit IRC14:57
*** kgiusti has joined #openstack-infra14:57
*** MCoLo has quit IRC14:57
*** krotscheck1 has joined #openstack-infra14:57
*** peristeri has quit IRC14:58
*** piet has joined #openstack-infra14:58
openstackgerritBeth Elwell proposed openstack-infra/storyboard: Amended documentation naming of installations  https://review.openstack.org/27466114:58
*** peristeri has joined #openstack-infra14:58
*** alivigni has quit IRC14:58
*** _amrith_ is now known as amrith14:59
*** daemontool has joined #openstack-infra14:59
jrollkrotscheck1: I know you've been working on mirror things, any thoughts on that ^14:59
krotscheck1jroll: The storyboard patch?15:00
openstackgerritIllia Khudoshyn proposed openstack-infra/project-config: Add gate job for Rally against Keystone v2  https://review.openstack.org/27466815:01
*** ajo has joined #openstack-infra15:01
krotscheck1jroll: Or the python-ironicclient problem?15:01
jrollkrotscheck1: the ironicclient thing15:01
krotscheck1jroll: It's likely because bandersnatch on mirror-update.openstack.org got stuck on a package.15:02
fungiAJaeger: we don't officially support any workflows involving git submodules and actively discourage their use in our environment, but i really don't know what happens if you push up a .gitmodules file in your repo and start trying to refer to other projects in gerrit with it. presumably it works the same way as the openstack/openstack repo is doing, though that one's really a special case15:02
jrollkrotscheck1: okay, how do we unwedge that15:02
krotscheck1jroll: I don't have the ACL's to troubleshoot that.15:02
*** amotoki has quit IRC15:02
jrollok15:02
krotscheck1jroll: I've already pinged jeblair to look into it once he gets online15:02
jrollkrotscheck1: awesome, thank you much15:02
krotscheck1jroll: It's all in backscroll.15:02
* krotscheck1 is rather annoyed that this irc client of his keeps disconnecting.15:03
jrollkrotscheck1: yeah, just doing the monday morning bump; our gate's down at the moment due to this15:04
*** [1]Thelo has joined #openstack-infra15:04
*** sridhar_ram has joined #openstack-infra15:04
krotscheck1fungi might be able to look into it. fungi do you have the time & expertise to poke at bandersnatch?15:04
krotscheck1jroll: Upside of the new mirrors, they're actually all in sync. Downside: They're actually all in sync.15:05
*** regXboi has joined #openstack-infra15:05
jrollheh15:05
openstackgerritJim Rollenhagen proposed openstack/requirements: python-ironicclient 1.1.0  https://review.openstack.org/27446215:06
*** Thelo has quit IRC15:07
*** [1]Thelo is now known as Thelo15:07
*** sigmavirus24_awa is now known as sigmavirus2415:09
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh  https://review.openstack.org/26711715:09
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Added wheel-release job  https://review.openstack.org/27354915:09
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror  https://review.openstack.org/16492715:09
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for kilo 2015.1.4 version bump bug 1540411  https://review.openstack.org/27470515:09
openstackbug 1540411 in neutron "kilo: ValueError: git history requires a target version of pbr.version.SemanticVersion(2015.1.4), but target version is pbr.version.SemanticVersion(2015.1.3)" [Critical,In progress] https://launchpad.net/bugs/1540411 - Assigned to Dave Walker (davewalker)15:09
*** NobodyCa1 has joined #openstack-infra15:09
*** BoneScratcher has joined #openstack-infra15:11
*** aysyd has quit IRC15:11
*** burgerk has joined #openstack-infra15:11
*** aysyd has joined #openstack-infra15:11
*** BoneScratcher has quit IRC15:12
*** placko has quit IRC15:12
AJaegerfungi, should we tell the salt guys - or wait and see?15:13
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add IRC notification for windmill  https://review.openstack.org/27469415:13
*** mriedem has joined #openstack-infra15:13
*** Daisy has joined #openstack-infra15:14
*** denisra_ is now known as denisra15:15
*** woodster_ has joined #openstack-infra15:16
*** rbrndt has joined #openstack-infra15:17
*** mohankumar_ has quit IRC15:17
pabelangerAJaeger: jeblair: Are we still good on landing 264994? Adds templating support to grafana for nodepool15:18
*** tiswanso has quit IRC15:18
fungikrotscheck1: jroll: in a meeting... summary?15:18
*** achanda has joined #openstack-infra15:18
*** Daisy has quit IRC15:18
krotscheck1fungi: python-ironicclient never made it to the AFS mirrors. Bandersnatch likely stuck on mirror-update.o.o. Ironic gate wedged as a result.15:19
openstackgerritMerged openstack-infra/storyboard: Amended documentation naming of installations  https://review.openstack.org/27466115:19
pabelangerAJaeger: did you hear anything more about Fedora nodes on Friday?15:19
pabelangerAJaeger: I didn't see what happen to them15:19
krotscheck1fungi: Amendment: python-ironicclient 1.1.0 was released yesterday. That particular version isn't on the mirrors.15:19
*** yamamoto has joined #openstack-infra15:20
AJaegerpabelanger: don't remeber hearing anything more15:21
AJaegerpabelanger: if jeblair is happy wieth 264994, so am I - but wanted him to +2 first since he had objections - to avoid accidental merge15:21
fungikrotscheck1: i was away during the implementation. what's the hostname where we're running bandersnatch now?15:22
anteayafungi: welcome back15:22
prometheanfiregreghaynes: https://review.openstack.org/#/c/273772/ and https://review.openstack.org/#/c/273795/ I think is all15:22
krotscheck1fungi: mirror-update.openstack.org15:22
krotscheck1I think15:23
pabelangerAJaeger: okay, will follow up with ianw again15:23
fungikrotscheck1: thanks15:23
*** dprince has joined #openstack-infra15:23
*** achanda has quit IRC15:23
fungianteaya: thanks, i was back on friday afternoon, though i'll admit i wasn't following things well since i was still catching up15:23
jrollthank you fungi, krotscheck115:24
anteayafungi: yup, I hear that15:24
*** followinfrared has quit IRC15:24
*** links has quit IRC15:24
anteayaI just finished the third-party meeting and have doubts about my performance today as I am not feeling well15:24
anteayagoing to have a nap and see how I feel15:24
openstackgerritMerged openstack-infra/elastic-recheck: Add query for kilo 2015.1.4 version bump bug 1540411  https://review.openstack.org/27470515:26
openstackbug 1540411 in neutron "kilo: ValueError: git history requires a target version of pbr.version.SemanticVersion(2015.1.4), but target version is pbr.version.SemanticVersion(2015.1.3)" [Critical,In progress] https://launchpad.net/bugs/1540411 - Assigned to Dave Walker (davewalker)15:26
AJaegeranteaya: get well soon!15:26
fungiinfra-root: looks like we have a stuck vldb lock. i'll try to start debugging when my current meeting wraps up, but this probably needs someone with deeper afs mojo than i possess (Bandersnatch completed successfully, running vos release. Could not lock the VLDB entry for the volume 536870931. VLDB: vldb entry is already locked)15:26
*** dprince has quit IRC15:26
*** EricGonczer_ has joined #openstack-infra15:27
*** Daisy has joined #openstack-infra15:27
nibalizerthats maybe related to the vos release I ran to add more replication to the pypi mirror15:27
krotscheck1...ahhhh15:28
ikhudoshynHi everybody! There is a small patch adding one more gate job for Rally https://review.openstack.org/#/c/274668 Could somebody please review it?15:30
*** achanda has joined #openstack-infra15:30
*** ayoung has quit IRC15:30
ikhudoshynIs it a right place to ask here?15:30
*** tiswanso has joined #openstack-infra15:31
funginibalizer: first occurrence in the log is from the 2016-01-30 00:40 utc run15:31
fungiso friday evening your time15:32
nibalizerhttp://paste.openstack.org/show/485622/15:37
sdaguethe lack of HP cloud is pretty noticable in the test load this morning. Bummer to be back at 600 nodes15:37
nibalizerI don't know how to tell if it is still proceeding or if its locked15:38
nibalizereventually the ssh I was running vos release in closed15:38
*** harlowja_at_home has joined #openstack-infra15:38
*** zeih has quit IRC15:39
*** mtanino has joined #openstack-infra15:39
*** zeih has joined #openstack-infra15:40
sdaguekrotscheck1 / fungi - the mirrors aren't serving wheels yet?15:40
*** achanda has quit IRC15:41
jeblairo/15:42
*** Sukhdev has joined #openstack-infra15:43
nibalizerfungi: it looks like maybe we could remsite the syncing thing15:44
jeblairnono15:44
jeblairdon't remsite15:44
jeblairit just needs to be continued15:44
nibalizerohai jeblair15:44
*** zeih has quit IRC15:44
jeblairi believe the initial copying of the data completed, but since vos release was interrupted, the flags weren't cleaned up15:45
*** dalgaaf has joined #openstack-infra15:45
openstackgerritZara proposed openstack-infra/storyboard-webclient: Index tags in search  https://review.openstack.org/27472315:45
jeblairif we urgently need updated pypi, we can switch the mirrors to serve the rw volume15:45
fungiit sounds pretty urgent. somehow (i'm unclear on the specifics) ironic's jobs have grown a dependency on an ironicclient release which happened over the weekend15:46
fungijroll can likely elaborate15:46
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Mark ironic-discoverd-ramdisk as deprecated in favor of ironic-agent  https://review.openstack.org/27026715:48
openstackgerritJames E. Blair proposed openstack-infra/system-config: Temporarily switch to AFS RW mirrors  https://review.openstack.org/27472915:49
jeblairwhile that's winding through, i'd love for someone to explain how a change landed that broke a gate.15:50
*** pradk has joined #openstack-infra15:50
fungijroll: ^ ?15:50
AJaegerAFAIR ironic merged a change that needs a new ironicclient release - and they were forced to release ironicclient to unblock their gate - and now it'S blocked since the new ironicclient client cannot get installed.15:50
AJaegerjroll: can you give more details, please?15:50
jeblairhow did "ironic merged a change that needs a new ironicclient release" happen?15:50
devanandawhat?15:51
jrolljeblair: so15:51
jrolldevstack removed keystone v215:51
devanandaI think this is the devstack thing15:51
AJaegerjeblair: I don't have full story, that was my impression from trying to get something related in...15:51
devanandaya, that15:51
jrollinstead of reverting that, we fixed our client to use v315:51
AJaegerAh...15:51
jroll(which is a thing that should happen anyway)15:51
stevemarjroll: devstack is reverting that change15:52
jrollsdague has ALSO proposed a revert15:52
jrollbut, we still want to release a client, regardless15:52
jrollwhich is done15:52
jrolland the requirements update is failing everything, because the new release is not in the mirrors15:52
jrollfungi: AJaeger jeblair ^ does that cover it?15:52
*** daemontool_ has joined #openstack-infra15:52
jeblairi understand why the requirements update is failing -- but what else is failing?15:52
fungimakes sense. so we should probably promote the revert in devstack too15:53
jrollrelevant logs: http://logs.openstack.org/62/274462/1/check/gate-tempest-dsvm-full/43c74f5/logs/devstacklog.txt.gz#_2016-02-01_12_12_09_60615:53
fungii'm guessing devstack changes aren't gated on ironic jobs15:53
jrolljeblair: ironic jobs that use old ironicclient that is keystone v2 only15:53
jrollcorrect15:53
krotscheck1sdague: nope.15:53
clarkbjroll your client should support both right?15:53
jrollclarkb: it should. it does in the release that went out this weekend.15:54
krotscheck1sdague: Eyeballs appreciated https://review.openstack.org/#/q/(topic:wheel_mirror)+status:open+branch:master15:54
devanandaclarkb: the changes do support both15:54
openstackgerritZara proposed openstack-infra/storyboard-webclient: Index tags in search  https://review.openstack.org/27472315:54
jrollclarkb: this release isn't in the mirrors and thus can't be put into g-r and u-c15:54
*** daemontool has quit IRC15:54
jrollbecause tests fail on the patch to do so15:54
AJaeger274462 is currently in the check queue but failing, needs a restart15:54
jeblairso the missing piece is the devstack-ironic check, which i imagine is intentionally missing.15:55
jrolljeblair: correct15:55
jeblairthis is an unfortunate consequence of that design15:55
jrollouru gate isn't stable enough to be there, unfortunately15:55
jrollour*15:55
jrollwe're working on it.15:55
jeblairnibalizer: want to +3 https://review.openstack.org/274729 ?15:56
pietAre any of the operator/admins in this channel interested in participating in a study on mobile use cases?  We're kind of hurting for folks to interview.  This is for the vahana project.15:56
*** salv-orlando has quit IRC15:57
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add allow-nonvoting key  https://review.openstack.org/27473515:58
*** zul has quit IRC15:58
*** rcernin has quit IRC15:58
jeblairfungi: i see you started looking into the vldb thing -- aiui, vos release 1) locks the volumes, 2) sets some flags on them, 3) copies the data, 4) unsets the flags, 5) unlocks the volume15:59
jeblairfungi: the vos release command was likely terminated during 3, so we're stuck at 415:59
nibalizerjeblair: done15:59
*** tswanson_ has quit IRC15:59
jeblairfungi: i believe there are forms of vos release that can continue the process, but i haven't use them yet15:59
fungithat makes sense. my 1500 meeting just wrapped so i'm taking a closer look at the log from friday now16:00
sdagueso the important thing to note, it didn't break "the gate". It broke some subset of projects outside the devstack in tree list.16:00
jeblairyeah, the 'gate' is why i didn't worry about this happening16:01
*** markmcclain has joined #openstack-infra16:01
*** Swanson has joined #openstack-infra16:01
jeblairfungi: (this is why the auto-releases haven't worked since friday -- that's when nibalizer started the release to the new site -- it was correctly locked while moving data)16:02
jeblairit only finished 10 hours ago16:02
fungioh!16:02
*** jsavak has quit IRC16:02
fungiwow, that did take a while16:02
fungithat was a local rsync between directories on the same filesystem?16:03
*** jsavak has joined #openstack-infra16:03
jeblairyeah; i'm thinking we might want to profile iad instead of ord as a potential second site16:03
*** mrmartin has joined #openstack-infra16:03
*** fawadkhaliq has quit IRC16:03
jeblairfungi: no, it was a full copy of the 230G volume from dfw to ord16:03
fungithat makes more sense16:03
jeblairstill not great, only got ~12mbit16:04
jeblairthere's room for improvement, but once the initial is done, the incrementals should be faster16:04
openstackgerritdaisy-ycguo proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite  https://review.openstack.org/27470116:05
fungimy math agrees with yours. that's definitely not a great sustained rate. i wonder if ssh was cpu-bound on one end?16:06
jeblairfungi: this was afs, not ssh16:06
fungioh, right, udp too?16:07
jeblairyup16:07
*** nmagnezi has quit IRC16:07
*** vgridnev has quit IRC16:07
*** abregman has quit IRC16:07
jeblairfungi: i'm looking into what we need to clean up16:09
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for magnum API test timeout  https://review.openstack.org/27474416:10
openstackgerritIllia Khudoshyn proposed openstack-infra/project-config: Add gate job for Rally against Keystone v2  https://review.openstack.org/27466816:13
openstackgerritMerged openstack-infra/system-config: Temporarily switch to AFS RW mirrors  https://review.openstack.org/27472916:14
odyssey4meAJaeger much cleaner, thanks to you: https://review.openstack.org/27272316:14
openstackgerritSteve Martinelli proposed openstack/requirements: bump oslo.config in upper constraints  https://review.openstack.org/27474816:15
*** mriedem is now known as mriedem_away16:15
openstackgerritMerged openstack-infra/elastic-recheck: Add allow-nonvoting key  https://review.openstack.org/27473516:15
*** krotscheck1 has quit IRC16:15
*** esp has quit IRC16:16
pabelangeryolanda: do you mind approving 274697 ? Missing node typo16:17
*** NobodyCam has quit IRC16:17
*** devananda has quit IRC16:17
*** jesusaurus has quit IRC16:18
*** zaro has quit IRC16:18
*** mrmartin has quit IRC16:18
*** samueldmq has quit IRC16:18
*** lxsli has quit IRC16:18
*** NobodyCa1 has quit IRC16:18
*** tcammann1 has quit IRC16:18
*** anteaya has quit IRC16:18
openstackgerritVictor Stinner proposed openstack/requirements: Bump oslo.context to 2.0 in upper constraints  https://review.openstack.org/27475816:18
*** cody-somerville- has quit IRC16:19
*** tcammann_ has quit IRC16:19
openstackgerritVictor Stinner proposed openstack/requirements: Bump oslo.concurrency to 3.4.0 in upper constraint  https://review.openstack.org/27476016:20
*** devananda has joined #openstack-infra16:22
*** fedexo has joined #openstack-infra16:22
*** tongli has joined #openstack-infra16:22
*** NobodyCam has joined #openstack-infra16:23
jeblairfungi: i'm going to unlock and release again (in screen this time)16:24
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: fix puppet lint errors  https://review.openstack.org/27476216:25
*** bpokorny has joined #openstack-infra16:26
realitixHello Infra. Can someone take a look at my gerrit account. I can't push because of contact form but I already did all the steps. My username: jsbevilacqua16:26
jeblairokay, probably should have grabbed the bandersnatch lock before doing that16:26
*** flwang1 has quit IRC16:28
*** tosky has quit IRC16:28
*** bpokorny has quit IRC16:28
*** _nadya_ has quit IRC16:28
*** bpokorny has joined #openstack-infra16:29
*** sbalukoff has quit IRC16:30
fungirealitix: it would be helpful to confirm that when you say "all the steps" you mean http://docs.openstack.org/infra/manual/developers.html#account-setup and not something you found on a random blog somewhere16:31
realitixOk I look at your guide and I answer to you in 2 minutes16:32
*** tosky has joined #openstack-infra16:32
*** jistr has quit IRC16:33
openstackgerritTim Birkett proposed openstack-infra/jenkins-job-builder: Add trigger for RabbitMQ trigger module  https://review.openstack.org/27383316:34
fungirealitix: for the record, i see two accounts in gerrit which look like they're probably both yours (so a duplicate account situation). the older one has username:realitix while the newer one is username:jsbevilacqua (both with the same gmail.com e-mail address)16:35
realitixYes16:36
realitixOk16:36
*** wei-wang has quit IRC16:36
realitixcan you merge the two account ?16:36
fungii can deactivate one and make it so you're only logging into the other, i just need to know which you want to use now16:37
realitixOK so i want to use realitix pseudo16:37
*** krotscheck1 has joined #openstack-infra16:37
ikhudoshynAJaeger: I've updated timeout for the job. I think 60 min would be enough https://review.openstack.org/#/c/274668/16:37
realitixnot jsbevilacqua16:37
krotscheck1Question: My current approach for wheel mirrors includes a daily execution of vos release - is that actually a sane thing to do given what we've just learned?16:38
fungirealitix: and right now your account id number at https://review.openstack.org/#/settings/ when you're logged in is 1880716:38
realitixIyes16:38
realitixIf you disable jsbevilacqua, have I to update my launchpad id ? (currently jsbevilacqua)16:38
openstackgerritdaisy-ycguo proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite  https://review.openstack.org/27470116:39
*** sigmavirus24 is now known as sigmavirus24_awa16:39
fungikrotscheck1: if the volume bandersnatch is writing to is the same volume the update workers write into, then we may need some lock waiting/retrying there16:39
fungirealitix: when i'm done you won't need to change anything in launchpad, no16:39
*** jlanoux has quit IRC16:39
realitixok great. Thanks16:39
*** rcernin has joined #openstack-infra16:40
*** thiagop has joined #openstack-infra16:40
krotscheck1fungi: I'm not certain what "volume" means in this context. The wheel mirrors are going to write to their own local afs disks, vos-release will be run on the release slave, bandersnatch is running on mirror-update...16:40
jeblairkrotscheck1, fungi: i think we should vos release immediately after each upload job16:40
*** shashank_hegde has joined #openstack-infra16:41
fungikrotscheck1: jeblair: right, we should release as close as possible to when we make changes to the content16:41
krotscheck1jeblair: On the wheel slave, or the release slave?16:41
*** sigmavirus24_awa is now known as sigmavirus2416:41
krotscheck1Or does it not matter?16:41
jeblairkrotscheck1: the release slave, since it requires admin permissions16:41
fungii didn't know if "vos release daily" meant "vos release at the end of our daily update job"16:42
*** daemontool has joined #openstack-infra16:42
krotscheck1Ok, so the build plan I have so far _should_ work16:42
*** salv-orlando has joined #openstack-infra16:42
*** baoli has quit IRC16:42
jeblairkrotscheck1: so it'd be a separate job run after the build one16:42
krotscheck1jeblair: Yep. https://review.openstack.org/#/c/273549/16:42
jeblairfungi: oh i see, yeah, so to generalize, each time we update a wheel mirror for a given arch, we should release that arch's wheel mirror16:42
jeblairkrotscheck1: ^16:42
*** baoli has joined #openstack-infra16:43
*** ashleighfarnham has joined #openstack-infra16:43
jeblair(whether that's daily or due to a specific change)16:43
krotscheck1Ok, I'm pretty certain my wheel patch chain does all those things.16:43
krotscheck1But AFS is still not quite done landing.16:43
jeblaircool, i'll try to review today as soon as i stop making more problems for myself16:43
* krotscheck1 goes off to check cacti for leftover usage.16:44
*** daemontool_ has quit IRC16:44
*** baoli has quit IRC16:46
realitixI have to go, thank you fungi for your help.16:47
fungirealitix: if you log out of gerrit and back in again you should be in the older "realitix" (10848) account now. the newer "jsbevilacqua" (18807) account has been deactivated16:47
realitixThanks16:47
krotscheck1jeblair, fungi, mordred: Looks like the old pypi mirrors' traffic has flatlines. We should be able to remove those now.16:47
openstackgerritMoshe Levi proposed openstack/diskimage-builder: Adding InfiniBand Support  https://review.openstack.org/27478116:47
krotscheck1*flatlined16:47
*** realitix has quit IRC16:47
jeblairkrotscheck1, fungi, mordred: let's leave them alone until i dig us out of the vos release mess i put us in, just in case something goes very wrong16:48
krotscheck1You got it16:48
clarkbfor the bluebox ovs gre tunnels I have confirmed that that underlying IP addrs are correct and that I can at least ping between them. So there isn't a general network issue between them16:50
*** HeOS has quit IRC16:50
*** zeih has joined #openstack-infra16:50
clarkbthe arp who hases are making it onto the ovs bridge but not across the gre tunnels16:51
clarkbwhich is why I had suspected something provider level previously16:51
*** harlowja_at_home has quit IRC16:51
krotscheck1Is the on-disk cacti stat for our mirrors accurate? Are we really only using 2-5GB of space for the afs cache?16:51
clarkbthe mtu for eth0 (the underlying itnerface) is 1500 and the mtu on the gre tunnel interfaces is 145016:52
*** ashleighfarnham has quit IRC16:52
*** mikelk has quit IRC16:54
jeblairkrotscheck1: which host?16:55
openstackgerritTim Birkett proposed openstack-infra/jenkins-job-builder: Add trigger for RabbitMQ trigger module  https://review.openstack.org/27383316:55
clarkbI hvae now confirmed that I have ip proto gre packets exiting eth0 on one host but not entering eth0 on the other16:55
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: added rspec files for automatic testing  https://review.openstack.org/27478816:56
clarkband the total length of those packets is far below any tiny mtu we may have16:56
*** baoli has joined #openstack-infra16:56
*** Daisy has quit IRC16:56
krotscheck1jeblair: All of them. bhs1.ovh is using 4 GB, dfw.rax is using 2.5 gra1 is using 3.2, etc etc16:57
*** jpr has joined #openstack-infra16:57
jeblairkrotscheck1: the rax ones have a cinder partition which wasn't being graphed in cacti16:57
krotscheck1jeblair: Ahhh, gotcha16:57
*** Swami has joined #openstack-infra16:57
jeblairkrotscheck1: where do you see 2.5 for dfw?16:57
* krotscheck1 idly wonders what the ratio of used/cached packages vs. actual mirror size is.16:58
krotscheck1jeblair: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=190 -> Used space /16:58
*** yamamoto has quit IRC16:58
clarkbI have double checked that our security groups are wide open so this must be filtering on the provider side16:58
jeblairkrotscheck1: ah.  yeah, so the used space on / is accurate for ovh... but includes everything, so we can probably guestimate it by subtracting 2.516:59
jeblairkrotscheck1: for rax, once http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=3172&rra_id=all populates, that will be accurate for just the afs cache16:59
*** tphummel has joined #openstack-infra17:00
clarkblet me double check our iptables but everything between the two nodes should be allowed17:00
*** EricGonczer_ has quit IRC17:00
fungiclarkb: it's the who-has to the ethernet broadcast address you're not receiving over the link?17:02
*** rcernin has quit IRC17:02
fungiclarkb: also i wouldn't suspect mtu anyway since those (even encapsulated) will be teensy17:03
*** bpokorny has quit IRC17:03
*** Apoorva has joined #openstack-infra17:04
*** annegentle has joined #openstack-infra17:04
clarkbfungi: yes, the arp who has for the address on the tunnel is being sent out eth0 encapsulated inthe tunnel. On the other side we don't see those packets17:04
clarkbI have zerod out the iptables counters and don't see any of the reject/drop rules going up with an active ping over the tunnel generating who has packet17:05
clarkbI suppose it could be ebtables17:05
fungiand the outer frame header and ip header destinations look correct?17:05
clarkbfungi: yes17:05
*** armax has joined #openstack-infra17:05
clarkb17:05:31.885684 fa:16:3e:cc:ab:a5 > fa:16:3e:32:51:39, ethertype IPv4 (0x0800), length 84: 10.220.2.161 > 10.220.2.160: GREv0, key=0x2, proto TEB (0x6558), length 50: 92:0d:cc:4f:86:42 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 172.24.4.1 tell 172.24.4.2, length 2817:05
*** vgridnev has joined #openstack-infra17:06
fungii wouldn't suspect ebtables or iptables if tcpdump shows the gre packet leaving the source and you also see it never leaving the destination17:06
clarkbthe two sets of IP addrs look correct to me17:06
fungiand yeah, that looks right. just making sure it wasn't strangely reusing the broadcast mac on the outer frame destination17:06
*** zz_dimtruck is now known as dimtruck17:07
*** EricGonczer_ has joined #openstack-infra17:07
fungithey've got to have some sort of upper-layer packet inspection going on, or some intermediary device mistakenly unwrapping the packet and trying to act on the payload17:08
*** ashleighfarnham has joined #openstack-infra17:08
*** JasonF is now known as JayF17:09
fungier, i meant to say earlier "...and you never see it arriving at the destination"17:09
clarkbjlk: if I leave that ping running can we use the above packet details to maybe track that down provider side17:09
*** esker has joined #openstack-infra17:09
clarkbfungi: right and it definitely isn't showing up on the interface with 10.220.2.16017:09
*** matrohon has quit IRC17:09
fungiclarkb: are you able to forward anything else over the tunnel? (like if you manually add a static arp entry on each end)?17:09
clarkbfungi: that is a good question, I haven't tried that17:10
fungiwould help to know if this is all gre not making it to the destination, or only arp over gre at issue17:10
clarkbya let me try that17:11
fungiahh, sorry, i was interpreting the description as the latter17:11
clarkbthat doesn't help either, just transitions ping to send icmp echo requests that are unanswered17:12
amaretskiyAJaeger hi17:12
clarkbarp/icmp similarly affected17:13
*** HeOS has joined #openstack-infra17:14
*** weshay has quit IRC17:14
amaretskiyAJaeger: recently we discussed issue with stuck requests logged in logs/devstack-gate-setup-workspace-new.txt.gz - there were periodical significant amount of lost time on this set-up17:15
*** tosky has quit IRC17:15
amaretskiyAJaeger: I've prepared a report with statistics (as you requested) - please look at http://amaretskiy.droppages.com/lost-seconds-while-setting-up-dsvm-workspace.html17:15
fungiclarkb: yep, so sounds more like there are filters on the provider's hosts or somewhere between which are discarding all ip 47 packets17:16
*** jgriffith_away is now known as jgriffith17:16
amaretskiyAJaeger: this report is based on small tool https://github.com/maretskiy/stuff/blob/master/openstack/parse_lost_seconds.py17:16
*** SumitNaiksatam has quit IRC17:16
amaretskiyplease look - in some cases there wee  11 minutes of lost time17:16
fungi(there is more to internet protocol than 1, 6 and 17!!!)17:16
amaretskiyAJaeger: ^17:16
*** gyee has joined #openstack-infra17:17
amaretskiyAJaeger: please contact me if you have any questions or additional data17:17
*** scheuran has quit IRC17:17
*** ifarkas has quit IRC17:17
*** HeOS has quit IRC17:19
*** rhallisey has joined #openstack-infra17:21
*** unicell has quit IRC17:21
fungiamaretskiy: that's pretty interesting stuff. we do track basic system statistics on our zuul mergers, e.g. http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1518&rra_id=all (there are 8 of them at the moment, zm01 through zm08). i wonder if the incidents line up with any load spikes on the corresponding merger worker mentioned in the fetch command?17:23
*** _bmjen is now known as bmjen17:23
*** cody-somerville has quit IRC17:23
jeblairslow fetches are probably due to too many refs17:25
clarkbI wonder if neutron has something that would prevent this by default17:25
clarkbarmax: if you have a chance, can you think of anything in neutron that would prevent running gre tunnels over the existing neutron tenant overlay networks?17:26
*** samuelBartel has quit IRC17:26
fungiamaretskiy: yeah, jeblair makes a good point. i would expect to see that more frequently on higher-volume projects (your example was for nova, i wonder if most incidents are on a small number of larger repos)17:27
armaxclarkb: you mean as in gre in gre?17:28
clarkbarmax: or gre in vxlan, yes17:28
fungigre between nova instances through unspecified neutron-based service provider's environment17:29
fungiwondering if they need some tuning of neutron in their environment to allow gre to pass between our instances17:30
clarkbthe odd thing is it works somtimes17:30
clarkbwhich makes me wonder if being on the same hypervisor makes it owrk17:30
clarkbbut once you actually traverse an overlay with an overlay it breaks17:31
armaxclarkb: I can’t think of anything it that could prevent that if not potentially resizing the MTU accordingly17:31
fungiclarkb: that would be my suspicion. you're seeing success between instances on the same host but failure between instances on different hosts17:31
fungiarmax: very small frames in this case, so wouldn't be mtu17:31
*** mgagne_ has quit IRC17:32
*** mgagne_ has joined #openstack-infra17:32
fungilike, the gre packets which aren't making it to their destination are an order of magnitude smaller than the expected mtu to begin with17:32
*** piet has quit IRC17:33
fungi(arp queries, non-padded icmp echo)17:33
*** _nadya_ has joined #openstack-infra17:33
*** mrmartin has joined #openstack-infra17:34
*** mgagne_ is now known as mgagne17:34
openstackgerritNikhil Komawar proposed openstack-infra/irc-meetings: Move Glance artifacts meeting  https://review.openstack.org/27480617:35
*** rhallisey has quit IRC17:36
*** salv-orl_ has joined #openstack-infra17:36
openstackgerritNikhil Komawar proposed openstack-infra/irc-meetings: Move Glance artifacts meeting  https://review.openstack.org/27480617:37
clarkbarmax: it is entirely possible that the provider installation is doing something funny and neutron isn't at fault. Was just curious if there was any known thing with neutron that might be at play (sounds like no)17:38
mgagneclarkb in a nutshell, what's up with gre? trying to setup gre within VMs?17:38
armaxclarkb: nothing on the top of my mind…perhaps it’s best to relay this question in #openstack-neutron (or the ML), as I might be missing something17:38
armaxclarkb: I’ll poke around17:39
clarkbmgagne: tl;dr is our multi node test envs use gre to set up an overlay network that tehy can control addressing on17:39
*** zeih has quit IRC17:39
fungigiven that we have what sounds like all gre passing between instances on the same host but not between instances on different hosts, it could be iptables on the hosts physical interfaces disallowing "scary" ip protocols, or similar filters somewhere in their underlying network deployment17:39
*** salv-orlando has quit IRC17:39
clarkbmgagne: and we see the gre packets exit host one but not enter host two17:40
*** yaume has quit IRC17:40
mgagneclarkb we found gre to not work properly unless nf_conntrack_proto_gre is loaded in the host17:40
jlkclarkb: okay17:40
mgagneclarkb even we wide open security gorups17:40
clarkbmgagne: on the hypervisor side you mean? let me check the vm side17:41
mgagneclarkb hypervisor side17:41
fungiwell, from the service provider perspective, it's that we see one virtual machine instance place the gre packets on its transmit buffer without error, but the destination virtual instance never sees it arrive (based on packet captures within the instances)17:41
*** shashank_hegde has quit IRC17:41
*** sjmc7 has joined #openstack-infra17:41
mgagneclarkb and you need to add GRE proto to sg, which I guess is already done unless it's done cross-cloud17:42
*** sjmc7 has left #openstack-infra17:42
clarkbmgagne: security groups are wide open17:42
mgagneclarkb 1) add gre proto to SG 2) ask provider to load gre conntrack helper unless they disabled SG completely17:42
mgagneclarkb for which proto?17:42
fungimgagne: that helps. i wasn't thinking about the fact that netfilter might need a plugin loaded to track gre state explicitly, since it's not a connection-oriented protocol17:43
mgagneclarkb tcp/udp/icmp?17:43
clarkbmgagne: ipv6, ipv617:43
*** zeih has joined #openstack-infra17:43
clarkber ipv4 and ipv617:43
mgagneI know for a fact that we didn't open all proto by default, only tcp/udp/icmp IIRC17:43
* fungi gets back on his "the internet is more than 1, 6 and 17" soapbox17:43
mgagne=)17:44
clarkbmgagne: I havne't looked at internap yet but it exhibits similar behavior17:44
clarkbmgagne: and if you don't allow gre well that would explain it17:44
mgagneyea, we know =)17:44
fungii guess that means no multihost devstack testing in at least some of our providers17:44
clarkbor convince our providers to allow ip17:45
jeblairmgagne: is there a reason you didn't?  how difficult would it be to alter?17:45
fungior do really ugly things like gre over udp17:45
fungiat least that would give our low-mtu testing a workout!17:45
clarkbthe underlying issue is that we need to give the inner cloud under test control of its own ip address space17:46
mgagnejeblair there is no built-in way to tell Neutron to use another set of default security group rules =)17:46
clarkbthere are other ways to tackle that problem17:46
fungimgagne: let me guess, your customers also don't have any interest in doing ipsec between their instances17:46
clarkbmgagne: oh you are saying just in neutron? we have that handled17:46
clarkbmgagne: the rule is allow all traffic between 0.0.0.0/0 and 0.0.0.0/017:47
clarkb(we don't set tcp/udp/icmp)17:47
*** _nadya_ has quit IRC17:47
mgagneclarkb was answering jeblair. we hacked neutron to override its default security group rules when creating default sg. we authorized udp/tcp/icmp by default since they are the common proto used. (sorry fungi)17:47
mgagneanyone can override them17:47
fungimgagne: yeah, if you're talking about our security groups, we override them to sane wide-open settings17:47
mgagneby "special" proto might need conntrack helper which Neutron won't load for you17:48
jeblairhave we done that on internap?17:48
clarkbjeblair: I have only confirmed bluebox so far17:48
jeblairperhaps we did not because we didn't think we needed to because tcp/udp appeared to be open17:48
jeblairclarkb: good, i remember doing that on bluebox17:48
jeblairso if no one else has set up internap rules, perhaps the solution is simple there17:49
mgagnelet me check. does it need to be default SG or do you create your own?17:49
*** mrmartin has quit IRC17:49
jeblairmgagne: (though even so, if your intent is to open things up by default (which i think is great), if you did so by opening all protocols, it might actually be simpler and better :)17:49
fungiright, and the bluebox situation could be as simple as getting the provider to load the netfilter connection tracking plugin for gre on their systems17:49
mgagnejeblair reviewing rules ATM17:50
jeblairmgagne: we can make our own if needed -- my suspicion is i just assumed i did not need too17:50
mgagnejeblair last time we setup them, it was in grizzly17:50
mgagneso things might have changed since17:50
clarkbfungi: we can also try switching to vxlan17:50
clarkbI will go ahead and push that patch and see what happens17:51
*** weshay_xchat has joined #openstack-infra17:51
mgagnefungi we should also asked neutron if they now load gre helper when gre is authorized in SG. but then, if you authorize ANY, they just can't load all helpers on earth just in case.17:51
jeblairoh that's an interesting idea17:51
*** jgriffith is now known as jgriffith_away17:51
*** weshay_xchat is now known as weshay17:51
jeblairclarkb: maybe we should explicitly authorize gre in bb?17:51
mgagneanyway, that's my take on gre not working. we did investigate when one customer asked about it and we find gre conntrack helper to be needed17:52
fungimgagne: it would probably make the most sense for neutron to make choosing conntrack helpers separate (or optionally separate) from ruleset writing17:52
*** aopincar has joined #openstack-infra17:52
mgagnefungi I don't have an opinion on how it should be done17:53
fungior better, make it possible to do proper "stateless" non-filtering17:53
* clarkb runs into the lack of documentation problem again17:54
fungibecause you only need conntrack to grok various protocols if you're trying to do stateful filtering. a stateful filter of all sources/destinations and all protocols/ports is pretty silly, just stop state tracking17:54
mgagnefungi it's hard =)17:54
fungiyeah, probably neutron's security groups need a proper "off switch" so tenants can opt out of having state tracking forced onto them17:55
*** jordanP has quit IRC17:55
mgagnefungi we did it in the past with our other line of products. we have a custom kernel and "forgot" to compile conntrack. now you want stateful filter, yea, can't load conntrack now, kernel doesn't know about it. And we didn't wish to reboot =)17:55
*** Sukhdev has quit IRC17:55
fungirather than having to make absurd "allow everything" filtering rulesets17:56
pabelangerfungi: do you have a moment to approve 274697? Fixes a typo for JJB17:56
mgagnefungi you cannot easily opt out of conntrack17:56
mgagnefungi you need to inject rules in the raw table instead of nat/filter17:56
fungithough stateless filtering support in neutron might also be interesting17:57
fungimgagne: yeah, obviously this would be for non-nat use cases17:57
*** cody-somerville- has joined #openstack-infra17:57
*** tcammann_ has joined #openstack-infra17:57
*** Sukhdev has joined #openstack-infra17:57
fungior, well, optionally for 1:1 nat use cases as well. just no layer 4 port address translation17:57
*** tcammann has joined #openstack-infra17:58
*** zaro has joined #openstack-infra17:58
*** hashar has quit IRC17:58
*** esp has joined #openstack-infra17:58
*** trown is now known as trown|lunch17:58
*** Sukhdev has quit IRC17:59
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Try vxlan instead of gre for overlays  https://review.openstack.org/27482118:00
*** Sukhdev has joined #openstack-infra18:00
*** Sukhdev has quit IRC18:00
clarkbthe options:key value passed to ovs-vsctl is the vxlan network identifier18:00
mgagnefungi will add to our backlog to do "real" any/any, not just tcp/udp/icmp18:03
mgagnewill only affect new users though18:03
*** deva_ has joined #openstack-infra18:04
*** unicell has joined #openstack-infra18:04
clarkbhttps://jenkins01.openstack.org/job/gate-tempest-dsvm-multinode-full/2522/ assuming I didn't get the vxlan stuff wrong that should tell us things18:04
*** jaosorior has joined #openstack-infra18:05
*** bhunter71 has joined #openstack-infra18:05
*** tcammann has quit IRC18:05
*** esp has quit IRC18:05
*** tcammann_ has quit IRC18:05
*** cody-somerville- has quit IRC18:05
*** zaro has quit IRC18:05
openstackgerritPavlo Shchelokovskyy proposed openstack-infra/project-config: Use Ironic as Devstack plugin in Rally gates  https://review.openstack.org/27482418:05
*** tcammann_ has joined #openstack-infra18:06
*** tcammann has joined #openstack-infra18:06
openstackgerritGraham Hayes proposed openstack/requirements: bump python-designateclient to 2.0.0  https://review.openstack.org/27368118:06
*** NobodyCa1 has joined #openstack-infra18:08
*** cody-somerville- has joined #openstack-infra18:08
*** deva_ has quit IRC18:09
*** deva_ has joined #openstack-infra18:09
*** hdd has joined #openstack-infra18:10
*** rubasov has quit IRC18:10
*** fedexo has quit IRC18:10
*** ashleighfarnham has quit IRC18:11
*** tcammann has quit IRC18:11
*** esp has joined #openstack-infra18:11
*** tcammann has joined #openstack-infra18:12
*** zeih has quit IRC18:13
*** NobodyCa1 has quit IRC18:13
openstackgerritMerged openstack-infra/project-config: Add missing node for windmill  https://review.openstack.org/27469718:14
*** deva_ has quit IRC18:14
*** ashleighfarnham has joined #openstack-infra18:14
dhellmannAJaeger : do we need a requirements repo change to go with the doc tools release? https://review.openstack.org/#/c/274461/118:14
*** NobodyCa1 has joined #openstack-infra18:14
*** Jeffrey4l has quit IRC18:14
*** yamahata has joined #openstack-infra18:14
*** cody-somerville- has quit IRC18:15
*** tcammann_ has quit IRC18:15
*** davideagnello has joined #openstack-infra18:15
*** esp has quit IRC18:15
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Re-Enable django_openstack_auth/designate-dashboard translations  https://review.openstack.org/27483218:16
*** jesusaurus has joined #openstack-infra18:16
*** tcammann_ has joined #openstack-infra18:16
*** dtantsur is now known as dtantsur|afk18:17
*** tcammann has quit IRC18:17
*** mrmartin has joined #openstack-infra18:18
openstackgerritPavlo Shchelokovskyy proposed openstack-infra/project-config: Use Ironic as Devstack plugin in Rally gates  https://review.openstack.org/27482418:18
*** zaro has joined #openstack-infra18:19
*** xyang1 has joined #openstack-infra18:19
clarkba quick spot check of the bluebox host that vxlaned shows that ping works18:20
*** NobodyCa1 has quit IRC18:20
clarkbwill have to see how tempest does18:20
*** NobodyCa1 has joined #openstack-infra18:20
AJaegerdhellmann: no, not needed. An update is only needed for openstack-manuals and that is not in requirements/projects.txt18:21
AJaegerdhellmann: thanks for double checking18:21
*** jesusaurus has quit IRC18:21
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: added spec files for automatic tests  https://review.openstack.org/27483518:21
clarkbsome RTTs were >2ms though which seems slow18:21
*** dims_ has joined #openstack-infra18:21
*** tcammann_ has quit IRC18:22
*** dims has quit IRC18:22
*** bpokorny has joined #openstack-infra18:22
clarkbusing the non tunnel addr is right around 1ms18:22
clarkbprobably not an issue for what we are doing18:22
*** yamahata has quit IRC18:23
*** jistr has joined #openstack-infra18:23
dhellmannAJaeger : ok, I'll process that now. Should we remove the existing entry from g-r?18:23
*** yamahata has joined #openstack-infra18:24
*** zaro has quit IRC18:24
*** NobodyCa1 has quit IRC18:25
*** sabeen3 has quit IRC18:26
AJaegeramaretskiy: thanks!18:26
*** baoli has quit IRC18:27
AJaegerdhellmann: no, leave it in since there's one repo that needs the tool (trove?) but version is not relevant18:29
dhellmannAJaeger : ok18:29
*** sridhar_ram has quit IRC18:29
*** mrmartin has quit IRC18:30
*** kushal has quit IRC18:31
*** rossella_s has quit IRC18:31
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite  https://review.openstack.org/27470118:31
*** kushal has joined #openstack-infra18:31
*** kushal has joined #openstack-infra18:31
*** rossella_s has joined #openstack-infra18:32
fungilooks like we have some _very_ old ready nodes (bare-trusty and devstack-trusty) on jenkins01 and jenkins04 from 18-26 days ago18:32
*** doug-fish has quit IRC18:32
fungiand some on other jenkins masters as old as 9 days18:33
fungijeblair: you wanted to hold the global lock on things happening on nodepool on friday... still the case? am i clear to nodepool delete those?18:34
fungijust trying to free up some job bandwidth18:34
*** jsavak has quit IRC18:35
*** jsavak has joined #openstack-infra18:35
fungigranted this would only free up ~4% of our overall quota, but nodepool also tends to not build as many new ready nodes if it sees these sitting around fulfilling demand18:36
*** doug-fish has joined #openstack-infra18:36
*** zeih has joined #openstack-infra18:37
AJaegerdhellmann: could you approve https://review.openstack.org/#/c/273088/? The release post job fails due to that currently (ok, failiure is cosmetic, still we should fix it to find real problems)18:37
*** shashank_hegde has joined #openstack-infra18:37
prometheanfirehow big are the various pools in nodepool?18:38
prometheanfirerackspace/hp/whoever18:38
*** _nadya_ has joined #openstack-infra18:38
dhellmannAJaeger : how can it be that we haven't given you +2 on that repo yet?18:38
dhellmannAJaeger : let me know if I should propose you, but +2a in the mean time18:38
fungiprometheanfire: see the max-servers lines in http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml18:39
fungiprometheanfire: that's pretty much always going to reflect the current situation18:39
prometheanfirefungi: thanks18:40
*** doug-fis_ has joined #openstack-infra18:40
AJaegerdhellmann: if you need another reviewer, feel free to propose me - but I won't make that repo a priority...18:40
fungiprometheanfire: we also have some per-provider trending at http://grafana.openstack.org/18:40
prometheanfireneato18:41
*** doug-fish has quit IRC18:41
*** doug-fis_ has quit IRC18:41
prometheanfireyou have a timeline on the osic stuff?18:41
*** jpr has quit IRC18:41
*** doug-fish has joined #openstack-infra18:41
*** jaosorior has quit IRC18:41
AJaegerdhellmann: thanks for release of openstack-doc-tools18:42
AJaegerjeblair, could you review 264994 and tell us whether that's fine now, please?18:42
*** ybathia has joined #openstack-infra18:44
*** rockyg has joined #openstack-infra18:45
*** AJaeger has quit IRC18:45
*** SumitNaiksatam has joined #openstack-infra18:45
*** doug-fish has quit IRC18:45
clarkbprometheanfire: they were supposed to get back to me mid last month but haven't heard from them since18:45
fungiprometheanfire: yep. in theory we've been using nodes provided by osic weeks ago?18:45
*** AJaeger has joined #openstack-infra18:46
clarkbI guess I should double check my email18:46
* fungi made a humor18:46
clarkbya nothing new on that thread18:46
*** AJaeger has quit IRC18:46
clarkbso we are just waiting18:46
fungibasically a couple months ago they said they've have something set up for us a couple weeks ago, but all's quiet on the western front18:47
*** AJaeger has joined #openstack-infra18:48
*** pvaneck has joined #openstack-infra18:48
*** jgriffith_away is now known as jgriffith18:48
*** edmondsw has quit IRC18:49
prometheanfireclarkb: fungi: I pinged a guy18:50
prometheanfirehopefully you'll get some status soon18:50
*** jistr has quit IRC18:51
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Re-Enable django_openstack_auth/designate-dashboard translations  https://review.openstack.org/27483218:51
fungiprometheanfire: appreciated. last we heard (per the committee conference calls) there was a defined order in which the first "customers" of the osic were being turned on. ours is #3 on the list, so i'm guessing it's just taking longer to set up #1 and #2 than expected18:51
*** trown|lunch is now known as trown18:52
*** followinfrared has joined #openstack-infra18:52
* prometheanfire shrugs, dunno the particular status, we just helped bring it online and handed it over more or less18:52
*** sabeen1 has joined #openstack-infra18:56
*** ihrachys has joined #openstack-infra18:59
jeblairfungi: i have released the global nodepool lock18:59
fungijeblair: appreciated. i'll clean up these moldy nodes19:00
*** jgriffith is now known as jgriffith_away19:00
*** salv-orl_ has quit IRC19:00
*** doug-fish has joined #openstack-infra19:01
jeblairShrews, notmorgan: i was chatting with mordred and shade/keystonev2 testing came up; he suggests changing the dsvm keystone v2 job on shade to use the stable/liberty branches19:02
openstackgerritMoshe Levi proposed openstack/diskimage-builder: Adding InfiniBand Support  https://review.openstack.org/27478119:02
notmorganjeblair: https://etherpad.openstack.org/p/v3-only-devstack good plan, we have a number of other things to hit too19:02
Shrewsjeblair: yeah, i think we should probably do that19:02
notmorganthere is brokeness in tempestlib, other functional tests too19:02
*** doug-fis_ has joined #openstack-infra19:04
*** jpr has joined #openstack-infra19:04
*** ihrachys has quit IRC19:04
fungiokay, removed 22 nodes which have been "ready" for more than a day. the only remaining "old" nodes now are a mere few hours. the oldest common node types in ready node are now close to none and have ages in seconds when they do exist19:04
*** ociuhandu has quit IRC19:04
*** doug-fis_ has quit IRC19:04
*** julim has quit IRC19:04
*** doug-fis_ has joined #openstack-infra19:04
fungiless common node types (tripleo, f23, et cetera) range up to 2-3 hours old19:05
*** doug-fish has quit IRC19:05
fungiwhich seems like our usual healthy situation19:05
*** krtaylor has quit IRC19:06
krotscheck1jeblair: I'm wrapping up here, how're things going with the afs lock?19:07
*** ddecapit has joined #openstack-infra19:07
*** ddecapit is now known as DuaneDeC719:07
pabelangerif there any issue with puppet being run on jenkins.o.o ?19:07
pabelangersorry, is*19:07
pabelangerthey seem to be unreported for 1h30m19:07
*** Daisy has joined #openstack-infra19:08
pabelangerHmm, seems like a lot more then just jenkins19:08
*** julim_ has joined #openstack-infra19:08
openstackgerritMartin Oemke proposed openstack-infra/project-config: Enable Translation for tacker  https://review.openstack.org/27485519:08
pabelanger112 nodes unreported19:08
*** zeih has quit IRC19:10
*** baoli has joined #openstack-infra19:12
*** julim_ has quit IRC19:12
*** Daisy has quit IRC19:12
*** krotscheck1 has quit IRC19:14
*** baoli_ has joined #openstack-infra19:15
*** sabeen3 has joined #openstack-infra19:16
*** aopincar has quit IRC19:16
*** krtaylor has joined #openstack-infra19:17
openstackgerritFrank Kloeker proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite  https://review.openstack.org/27470119:18
*** baoli has quit IRC19:18
*** kzaitsev_mb has quit IRC19:18
*** sabeen1 has quit IRC19:19
*** julim has joined #openstack-infra19:20
*** marekd has quit IRC19:21
pabelangerfungi: happen to know what is going on with puppet? ^19:22
clarkbhttps://jenkins01.openstack.org/job/gate-tempest-dsvm-multinode-full/2522/console appears to have lost jenkins connectivity19:23
clarkbso vxlan may not work well either (or it could have been a fluke)19:23
AJaegersdague: could you review a new job for novaclient, please? https://review.openstack.org/26695519:23
clarkbwe should see if the neutron test fails the same way19:24
notmorganfungi, jeblair, clarkb: where is the convo re phabricator happening? (cc zzzeek )19:26
clarkbnotmorgan: not sure I understand the context19:27
notmorganclarkb: for issue tracking, just what is the status/where is that all happening (infra meetings? infra ml?, etherpads?)19:27
*** davideagnello has quit IRC19:28
*** nikhil_k has joined #openstack-infra19:28
notmorganclarkb: zzzeek and i were talking about projects and moving them to use gerrit but LP being the issue tracker is rough.19:28
clarkbnotmorgan: craige has been working it and keeping us up to date in the infra meetings19:28
notmorganclarkb: because gerrit workflow is great :) [*spoiled*]19:28
notmorganclarkb: fantastic :)19:29
fungipabelanger: i'll check our main log19:31
*** annegentle has quit IRC19:31
*** nikhil has quit IRC19:31
funginotmorgan: also to be more exact, it's maniphest which is the task tracker (from the phabricator suite of systems)19:31
*** annegentle has joined #openstack-infra19:31
*** marekd has joined #openstack-infra19:32
*** edmondsw has joined #openstack-infra19:33
fungiclarkb: i guess 264372 can be abandoned?19:33
*** ociuhandu has joined #openstack-infra19:33
clarkbya I will clean that up19:34
notmorganfungi: ok sure! :)19:34
notmorganbut still :)19:34
notmorgangood to know19:34
*** jpr has quit IRC19:34
clarkbfungi: done19:34
funginotmorgan: when you say "phabricator" a lot of people will assume you mean their code review system19:34
notmorganright19:35
*** jsavak has quit IRC19:35
notmorganmy mistake19:35
notmorgan;)19:35
* notmorgan corrects brain mapping of name of things19:35
*** jsavak has joined #openstack-infra19:35
pabelangerfungi: thanks19:35
jeblairkrotscheck1 appears to have disappeared19:36
*** hashar has joined #openstack-infra19:36
jeblairbut the answer is that i think it will take me a while to get the bandersnatch RO mirror going, but in the mean time, the RW should be fine, and none of that should have any impact on the wheel work19:36
clarkbthe neutron job with vxlan for the floating ip overlay succeeded19:37
jeblairaside from the fact that i haven't had time to review wheel-related patches19:37
*** Sukhdev has joined #openstack-infra19:37
jeblairi'm catching up on stuff i neglected last week19:37
clarkbhttps://jenkins03.openstack.org/job/gate-tempest-dsvm-multinode-full/2665/console is running in rax-dfw now instead of on bluebox will have to see if the network trouble is consistent19:37
jeblairclarkb: once you're done with vxlan testing, do you want to test the security group changes with gre?19:37
clarkbjeblair: what changes? I thought we were already good?19:38
clarkbbluebox allows all ipv4 and ipv619:38
clarkbaccording to the api at least19:38
jeblairclarkb: my understanding of the conversation earlier was that in internap we may have success in creating our own security group because the default one does not allow gre19:38
jeblairclarkb: and there was also the theory that perhaps if we specifically allowed gre in bluebox, it might trigger a conntrack module load19:39
clarkbah19:39
jeblairclarkb: (and i don't know the sg state in ovh; one or both of those may be applicable there too?)19:40
clarkbI missed that19:40
clarkbok looking at internap we do have explicit tcp/icmp/udp19:40
*** Sukhdev has quit IRC19:40
clarkbdoes neutron api accept an arbitrary IP type value? /me reads docs19:40
jeblairclarkb: we might also try 'all' in internap19:41
*** piet has joined #openstack-infra19:43
clarkbdoesn't look like the api accepts arbitrary ip types19:43
*** hashar has quit IRC19:43
clarkbnull, icmp, icmpv6, tcp, and udp are the values according to the docs19:43
clarkbso I don't think we can add a rule in bluebox for gre19:44
jeblairok19:44
*** gongysh has joined #openstack-infra19:44
clarkbI can however try switching the rules in internap to match the others which use null and should apply to all types19:44
jeblair++19:44
clarkbwill do that now19:45
*** mrmartin has joined #openstack-infra19:45
*** matrohon has joined #openstack-infra19:45
notmorganclarkb, fungi: if we have a project that uses an external issue tracker, is there oposition to using a script provided to infra to update issue tracker? or do i need to watch the gerrit stream and source the data from there?19:47
notmorganclarkb, fungi: until manaphest is ready and then there could be a move to that [evaluated when it's ready]19:47
clarkbnotmorgan: I would not want to be responsible for maintaining arbitrary bug tracker updates19:48
jeblairnotmorgan: we talking official or unofficial project?19:48
clarkb(more details may help)19:48
notmorganjeblair, clarkb: looking at possibvly moving dogpile (and then talking about SQL-A depending on zzzeek's happiness with it all) to gerrit19:48
jeblairso probably not wanting to become an official openstack project19:49
notmorganright19:49
* notmorgan just agreed to help co-maintain dogpile so there are more eyes / help on it19:49
*** krotscheck1 has joined #openstack-infra19:50
*** flepied has quit IRC19:50
notmorganand yes i am totally spoiled by gerrit workflow vs PRs.19:50
notmorgansooooooo19:50
notmorgan:)19:50
notmorgani'm totally fine if it should be external bot reading the stream.19:51
clarkbjeblair: ok I have added two rules to internap, one for ipv4 0.0.0.0/0 and one for ipv6 ::/0 with all other details left blank19:51
clarkbjeblair: should go ahead and delete the icmp/tcp/udp rules?19:52
clarkbnotmorgan: so these use the github tracker?19:52
notmorganclarkb: right now i think one uses bitbucket, the other github.19:52
*** cody-somerville has joined #openstack-infra19:52
*** cody-somerville has quit IRC19:52
*** cody-somerville has joined #openstack-infra19:52
notmorganbut essentially same kind of thing.19:52
*** flepied has joined #openstack-infra19:53
jeblairclarkb: yes, i think delete the rules19:53
jeblairclarkb: (mostly for cleanliness; i think theoretically it shouldn't matter)19:54
Shrewsjeblair: is there a way to lock a job down to a certain version of devstack? or is that done for you automatically if you lock it to stable/liberty?19:54
clarkbjeblair: all done19:55
*** ihrachys has joined #openstack-infra19:55
clarkbnotmorgan: the first thing to keep in mind is we can really only sanely have one set of commentlinks in the gerrit config so the turn string in commit message to link to github won't happen. Instead it will be a broken link to launchpad19:56
*** jgriffith_away is now known as jgriffith19:56
notmorganclarkb: i was actually thinking of having a bot add a separate link and use a different link-name for these, just not auto-linked in the gerrit css19:56
clarkbnotmorgan: the problem with that is you break all those links when you switch to maniphest19:57
clarkbnotmorgan: because you have changed the string19:57
notmorganclarkb: well, i'll keep chatting with zzzeek on this and figure it out. also will bug craige and see where we are19:57
notmorgancause if it's "soon-ish" then it can wait :)19:58
notmorganclarkb: thanks!19:58
*** jsavak has quit IRC19:59
*** jsavak has joined #openstack-infra20:00
clarkbnotmorgan: also github goes down a lot and aggressively rate limits us so there are various implementation details that you may have to work around20:00
clarkbI haven't tried interfacing with bitbucket to know if it has similar problems20:00
notmorganclarkb: possibly20:00
*** hashar has joined #openstack-infra20:01
*** amitgandhinz has quit IRC20:03
*** salv-orlando has joined #openstack-infra20:03
*** amitgandhinz has joined #openstack-infra20:04
*** weshay has quit IRC20:05
*** mrmartin has quit IRC20:05
*** salv-orlando has quit IRC20:07
*** annegentle has quit IRC20:11
prometheanfireclarkb: can I get a email or something to give to the osic guy so he can search for you?20:12
prometheanfirefungi: ^ you too :P20:13
clarkbprometheanfire: cboylan@sapwetik.org is what I was on the thread as20:13
prometheanfirethanks20:14
fungiback now, sorry, quarterly pest control visit20:14
*** sbalukoff has joined #openstack-infra20:14
*** ldnunes has quit IRC20:16
*** weshay has joined #openstack-infra20:17
fungiand yeah, clarkb graciously volunteered to be the main point of contact on the osic thing20:17
prometheanfireclarkb: he found you20:18
notmorgankrotscheck1: ping20:18
*** annegentle has joined #openstack-infra20:20
openstackgerritDavid Shrewsbury proposed openstack-infra/project-config: Change shade keystone v2 job for stable/liberty  https://review.openstack.org/27487720:22
Shrewsjeblair: mordred: ^^^^20:22
*** jsavak has quit IRC20:22
* Shrews hopes that's all that is needed20:23
*** jsavak has joined #openstack-infra20:23
*** baoli_ has quit IRC20:23
*** nihilifer has quit IRC20:26
*** tongli has quit IRC20:28
Shrewssdague: also fyi per our convo this morning ^^^20:29
*** nihilifer has joined #openstack-infra20:30
AJaegersdague: one more review request for you - I think you added this initially: https://review.openstack.org/#/c/272411/20:31
*** markmcclain has quit IRC20:32
*** rockyg has left #openstack-infra20:32
*** markmcclain has joined #openstack-infra20:33
*** DuaneDeC7 has quit IRC20:34
*** jsavak has quit IRC20:35
*** jsavak has joined #openstack-infra20:36
*** electrofelix has quit IRC20:37
*** flwang1 has joined #openstack-infra20:37
*** DuaneDeC7 has joined #openstack-infra20:38
*** gongysh has quit IRC20:38
*** _nadya_ has quit IRC20:40
*** ashleighfarnham has quit IRC20:40
*** DuaneDeC7 has quit IRC20:44
*** DuaneDeC7 has joined #openstack-infra20:44
*** baoli has joined #openstack-infra20:45
*** baoli has quit IRC20:45
*** baoli has joined #openstack-infra20:46
*** followinfrared has quit IRC20:46
*** timothyb89_ is now known as timothyb8920:47
*** doug-fis_ has quit IRC20:47
*** ayoung has joined #openstack-infra20:47
*** dtardivel has quit IRC20:48
*** zeih has joined #openstack-infra20:48
*** doug-fish has joined #openstack-infra20:48
pabelangerfungi: things look better on puppetdb now, what was the issue?20:48
*** sabeen3 has quit IRC20:49
fungipabelanger: a 130-minute delay before saying...20:50
fungi2016-02-01 20:15:28,471 p=1604 u=root |  fatal: [mirror.region-b.geo-1.hpcloud.openstack.org]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}20:50
fungilooks like we need to dereigster that20:51
*** Apoorva has quit IRC20:51
jeblairwonder why the cache didn't update?20:51
pabelangerfungi: wow, 130min timeout20:51
fungiyeah, i don't see mirror.region-b.geo-1.hpcloud.openstack.org mentioned in system-config20:52
*** salv-orlando has joined #openstack-infra20:53
bknudsondo you know why, when tox -e docs in infra, the remote.origin.url would have something like p/openstack/keystone rather than openstack/keystone?20:53
*** thomasem is now known as thomasem-away20:53
bknudsontrying to figure out why all the project source links are wrong in the developer docs.20:53
jeblairfungi: we're using the openstack ansible inventory20:53
jeblairfungi: it caches because it's slow20:53
jeblairfungi: so it's probably still in the cache20:53
clarkbbknudson: we dont use tox -e docs20:53
fungiright, just making sure we'd thoroughly removed it from our static lists before looking into the caching20:54
*** Sukhdev has joined #openstack-infra20:54
jeblairok20:54
fungiwhich we seem to have done20:54
clarkbbknudson: but the p/ prefix is a gerritism20:54
jeblairfungi: so i think either it disappeared since the last cache update, or maybe if it can't contact the cloud at all, it doesn't update the cache20:54
bknudsonthe cgit link doesn't have p/ -- http://git.openstack.org/cgit/openstack/keystone20:54
clarkbbknudson: correct it is a gerritism20:55
jeblairfungi: Shrews may either know or be interested in the answer :)20:55
AJaegerbknudson: we use "python setup.py build_sphinx"20:55
*** hichihara has joined #openstack-infra20:55
fungii need to educate myself on what updates the /var/cache/ansible-inventory/ansible-inventory.cache file, but first i need to step away again so please don't let me hold up someone else investigating20:55
jeblairi am not investigating20:55
Shrews??20:55
fungii'll continue looking in a bit when i get back, if nobody beats me to the root cause20:56
bknudsonok, I thought it did tox -e docs, but either way oslosphinx is doing `git config --local --get remote.origin.url` which has a p/ in the path.20:56
jeblairShrews: hpcloud disappeared, but a host is still in our ansible inventory cache; i'm wondering if ansible will eventually remove it, or will it stay there forever since it can't contact the cloud at all?20:56
fungiftr, mirror.region-b.geo-1.hpcloud.openstack.org is definitely still in /var/cache/ansible-inventory/ansible-inventory.cache and that file was last modified 18.5 hours ago20:57
*** jcoufal has quit IRC20:57
ianwclarkb: you moved the nodepool logging config over to the template, did we update the generation scripts?20:57
fungiso if it's a daily cache update, that would 'splain it i guess20:57
clarkbianw: I did20:57
* fungi will bbiab20:57
jeblairfungi: yeah, and i don't know when the host actually disappeared...maybe cacti does20:57
*** hdd has quit IRC20:57
*** ashleighfarnham has joined #openstack-infra20:57
clarkbianw: just needed path changes otherwise should be the same20:57
AJaegerCould some project-config cores review the next translation step, please? https://review.openstack.org/#/c/274421 switches most repos to the new set up20:57
*** kzaitsev_mb has joined #openstack-infra20:58
jeblairfungi, Shrews: we only lost contact with the host a few hours ago; perhaps the cloud went down at the same time.  so yeah, we probably have not exercised the cache update since then.20:58
openstackgerritMikhail S Medvedev proposed openstack-infra/devstack-gate: Use first index-url in pip.conf to derive pypi_host  https://review.openstack.org/27488621:00
zeihAJaeger: should the patch be abandoned or workflow -1? https://review.openstack.org/#/c/274855/21:01
Shrewsjeblair: fungi: there is a refresh option to the inventory, if that's useful for you21:01
*** dims has joined #openstack-infra21:02
openstackgerritIan Wienand proposed openstack-infra/system-config: Update nodepool logging for Fedora 23  https://review.openstack.org/24790921:02
ianwclarkb: ^ ok, the file there seems older ... jeblair had an update out there that got most of these changes, but maybe it's lost in the swizzle around21:02
AJaegerzeih: abandon it - you can always restore. And it's easy to regenerate...21:02
zeihAJaeger: yes it is. okay. Have a nice evening21:03
*** dims_ has quit IRC21:03
clarkbianw: I didn't update any existing changes that update the file, I just updated the generation script21:04
ianwclarkb: right, digging further, the change i was thinking of didn't merge -> https://review.openstack.org/#/c/272757/ .  that explains why the file looks "old"21:04
*** zeih has quit IRC21:04
clarkbianw: I think the change that that conflicts with is the one I wrote which does the more complete thing21:05
clarkbmaybe?21:06
*** unicell has quit IRC21:06
*** vivekd has joined #openstack-infra21:06
clarkboh wait I see21:06
clarkbwe just need to update to pick up delta gotcha21:06
clarkbso that will need an update and modify the imgae builder logging config instead21:06
*** jsavak has quit IRC21:06
*** doug-fish has quit IRC21:06
*** sridhar_ram has joined #openstack-infra21:07
*** jsavak has joined #openstack-infra21:07
ianwclarkb: https://review.openstack.org/#/c/247909/ i just updated will do that ... it's been around since nov 2021:07
*** baoli has quit IRC21:07
ianwmaybe we can just merge it, and we can abandon these others21:07
*** doug-fish has joined #openstack-infra21:07
*** baoli has joined #openstack-infra21:07
clarkbianw: ++21:08
clarkbianw: +2'd21:08
*** baoli has quit IRC21:08
*** rocky_g has joined #openstack-infra21:08
*** baoli has joined #openstack-infra21:09
*** esker has quit IRC21:09
*** markmcclain has quit IRC21:09
*** doug-fis_ has joined #openstack-infra21:09
*** markmcclain has joined #openstack-infra21:09
*** davideagnello has joined #openstack-infra21:10
*** doug-fi__ has joined #openstack-infra21:11
*** hichihara has quit IRC21:11
*** doug-fish has quit IRC21:11
*** thorst has quit IRC21:12
clarkbany objection to me asking emc ipsilon third party ci to stop checking devstack-gate?21:13
*** Sukhdev has quit IRC21:13
*** doug-fis_ has quit IRC21:13
jeblairclarkb: huh, i thought i disabled them21:13
jeblairon friday21:14
clarkbjeblair: well that would work too I suppose but they just commented on my vxlan change21:14
clarkbperhaps they were reenabled or have multiple accounts? as stackalytics shows making a new account is apparently a work around of choice around here21:15
*** Apoorva has joined #openstack-infra21:15
*** doug-fi__ has quit IRC21:15
*** ZZelle_ has joined #openstack-infra21:15
clarkbI rechecked the vxlan change to get it to run against more clouds and see if I can tickle the jenkins disconnect again or maybe that was just a fluke21:15
jeblairnope, same account21:16
*** rcernin has joined #openstack-infra21:19
*** darrenc_ is now known as darrenc21:19
*** aysyd has quit IRC21:20
openstackgerritMerged openstack-infra/system-config: Update nodepool logging for Fedora 23  https://review.openstack.org/24790921:21
*** doug-fish has joined #openstack-infra21:23
*** baoli has quit IRC21:24
dansmithI'm getting lots of 503s from gerrit just now21:24
jeblairclarkb: the account is still disabled21:25
clarkbjeblair: huh21:25
clarkbdansmith: I think I have tracked that down to paache deciding the backend has gone to lunch and thus refusing to ocnnect to it21:26
clarkbbut I havne't figured out why apache thinks that yet21:26
*** jsavak has quit IRC21:26
jeblairclarkb: they have some open connections though, so i'll kill them21:26
dansmithclarkb: yeah, apache will do that for some cache interval if it gets something seemingly fatal from the backend21:26
stevemarsdague: just finished documenting a bunch of failures on the etherpad https://etherpad.openstack.org/p/v3-only-devstack21:26
*** jsavak has joined #openstack-infra21:27
dansmithclarkb: just thought maybe it was indicative of time-to-reboot-gerrit21:27
clarkbdansmith: it may be21:27
openstackgerritMerged openstack-infra/system-config: Add storyboard.openstack.org to cacti  https://review.openstack.org/27417721:27
*** doug-fis_ has joined #openstack-infra21:27
*** markmcclain has quit IRC21:27
*** doug-fish has quit IRC21:28
*** julim has quit IRC21:28
*** sabeen has joined #openstack-infra21:28
*** dtardivel has joined #openstack-infra21:29
*** amrith is now known as _amrith_21:31
*** thorst has joined #openstack-infra21:31
*** doug-fis_ has quit IRC21:32
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add functional tests for windmill  https://review.openstack.org/27490021:32
*** doug-fish has joined #openstack-infra21:33
*** DuaneDeC7 has quit IRC21:34
*** markmcclain has joined #openstack-infra21:35
*** sabeen2 has joined #openstack-infra21:35
*** esker has joined #openstack-infra21:37
*** esker has quit IRC21:37
*** esker has joined #openstack-infra21:37
*** sabeen has quit IRC21:37
*** MarkAtwood has quit IRC21:41
openstackgerritMerged openstack/requirements: remove argparse from requirements list  https://review.openstack.org/27035421:42
*** rcernin has quit IRC21:44
*** matrohon has quit IRC21:45
*** Sukhdev has joined #openstack-infra21:47
openstackgerritMathieu Gagné proposed openstack-infra/system-config: gerrit: Add borders to test result table  https://review.openstack.org/26386621:47
*** jpr has joined #openstack-infra21:48
clarkbinfra-root osic has asked me for a list of users and their email addrs, should I go ahead and tell them seomthing like admin@openstack or similar?21:49
*** eil397 has joined #openstack-infra21:49
*** peristeri has quit IRC21:50
*** edmondsw has quit IRC21:50
openstackgerritMathieu Gagné proposed openstack-infra/system-config: gerrit: Restore reasonable max-width to change info table  https://review.openstack.org/26782321:51
clarkbjeblair: how did you do that for bluebox and ovh and internap? I can try to keep them all similar21:51
*** angdraug has joined #openstack-infra21:52
*** burgerk has quit IRC21:56
*** devananda has quit IRC21:57
*** rossella_s has quit IRC21:58
*** ihrachys has quit IRC22:03
*** j_king has quit IRC22:05
*** amitgandhinz has quit IRC22:06
*** j_king has joined #openstack-infra22:06
*** erlon has quit IRC22:06
*** eharney has quit IRC22:06
*** amitgandhinz has joined #openstack-infra22:07
*** thomasem-away is now known as thomasem22:09
*** vgridnev has quit IRC22:09
*** chlong has joined #openstack-infra22:10
*** thomasem is now known as thomasem-away22:15
jeblairclarkb: i think the password file lists all of that22:15
openstackgerritIan Wienand proposed openstack/diskimage-builder: Add support for gentoo to simple-init  https://review.openstack.org/27379022:15
jeblairkrotscheck1: i know you're away now, but hopefully i left enough comments on the wheel patches that you can update them and maybe we can merge them tomorrow22:16
jeblairfungi: you may want to review these as well (topic:wheel_mirror) and related22:16
jeblairkrotscheck1: i haven't made the volumes yet, i'll do that tomorrow after i confirm the names and paths with you (that's mentioned in one of the comments)22:17
ianwgreghaynes: have you looked at this centos build failure in dib?  just don't want to waste time rediscovering the same things ... :)22:17
prometheanfirekik22:17
*** Sukhdev has quit IRC22:18
*** EricGonczer_ has quit IRC22:18
*** rfolco has quit IRC22:19
*** Sukhdev has joined #openstack-infra22:20
openstackgerritIan Wienand proposed openstack/diskimage-builder: Fix package-installs for python3  https://review.openstack.org/27378222:22
*** Sukhdev has quit IRC22:22
prometheanfirenice to see ianw and greghaynes taking over my reviews :|22:22
*** piet has quit IRC22:24
ianwgreghaynes: hmm, i wonder if this centos6 build is even worth testing in CI ...22:29
*** Sukhdev has joined #openstack-infra22:31
*** Sukhdev has quit IRC22:31
*** Sukhdev has joined #openstack-infra22:31
openstackgerritMerged openstack/requirements: python-ironicclient 1.1.0  https://review.openstack.org/27446222:32
krotscheck1jeblair: Thanks, I'll take al ook22:36
*** trown is now known as trown|outttypeww22:36
clarkbjeblair: looks like we used infra-root@openstack.org which I am guessing is an alias for all of us22:37
clarkbI will use that22:37
*** tiswanso has quit IRC22:38
clarkboh we also have openstack-infra@openstack.org22:38
greghaynesianw: ya, if its failing a lot SGTM22:38
clarkbI guess I hsould do some digging to determine which is more appropriate22:38
*** Sukhdev has quit IRC22:38
*** ybathia has quit IRC22:38
ianwgreghaynes: something borked with packages i guess22:39
ianw2016-02-01 21:10:04.260 | Error: Package: python2-rsa-3.3-2.el6.noarch (epel)22:39
ianw2016-02-01 21:10:04.260 |            Requires: python-pyasn1 >= 0.1.322:39
ianw2016-02-01 21:10:04.260 |            Available: python-pyasn1-0.0.12a-1.el6.noarch (base)22:39
openstackgerritMerged openstack/requirements: tuskar is no longer maintained  https://review.openstack.org/27308822:39
ianwtrying to get to the bottom of it first ...22:39
greghaynesianw: oh, so its an actual break22:39
ianwmaybe, but just as likely we're doing something silly to cause it...22:39
*** daemontool has quit IRC22:39
*** devananda has joined #openstack-infra22:39
Sam-I-Amanyone here know about the docs ftp publisher?22:41
ianwi'm just bringing up a centos6 vm to test it in ... been (thankfully) long time since i had to use one :)22:41
Sam-I-Amor... can help me with a potential problem22:41
clarkbSam-I-Am: I know things22:41
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Fix on user idenfier contruction  https://review.openstack.org/27491722:41
Sam-I-Amclarkb: cool22:41
Sam-I-Amclarkb: it looks to me that it doesn't delete obsolete files?22:42
*** camunoz has joined #openstack-infra22:42
clarkbSam-I-Am: correct, its just ftp22:42
fungiokay, back again. catching up and then digging into ansible inventory cache questions22:42
Sam-I-Amclarkb: hmm, that is a problem22:42
clarkbfungi: do you know what the differences between infra-root@openstack.org and openstack-infra@openstack.org are? we used one on ovh and the other on internap22:42
Sam-I-Amclarkb: google picks up on individual files and people can still get to them, even when the docs no longer reference and delete them22:42
clarkbSam-I-Am: yes that is a long known problem with cloud sites22:43
*** Sukhdev has joined #openstack-infra22:43
clarkb(if we could rsync this would be a non issue)22:43
Sam-I-Amis there any way to remove the target dir before copying files?22:43
*** amitgandhinz has quit IRC22:43
Sam-I-Amor is there a job that does cleanup that i can trigger22:43
fungijeblair: krotscheck1: thanks--some light evening reading!22:44
clarkbcurrently no, I think it has to be done manually (I don't know that jenkins' publisher can express deletes)22:44
Sam-I-Amclarkb: hmm, that's frustrating. so things just keep piling up?22:44
clarkbSam-I-Am: yes, it is one of the things that will be addressed by moving the hosting of docs but I don't think anyone is actively working that yet22:46
Sam-I-Amclarkb: is there a way i can submit a list of files to remove to... someone?22:47
clarkbSam-I-Am: I think annegentle had been cleaning them up in the past? might be worth starting there to make sure we aren't doing something that has a process already22:47
Sam-I-Amok, i'll ask her22:47
Sam-I-Amthx22:47
fungiclarkb: i definitely do not know the difference between those two mail aliases, have you logged into their inboxes? are they going to the same account?22:47
*** jpr has quit IRC22:48
annegentleSam-I-Am: clarkb: we'd like to have infra implement swift objects so that the manual process can be dropped.22:48
clarkbfungi: I havne't yet am about to as soon as I get power to laptop22:48
Sam-I-Amand there she is22:48
annegentlefungi: I've asked around about prioritizing this spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/doc-publishing.html22:48
annegentlefungi: can you try to find a resource?22:48
annegentleSam-I-Am: log a bug and assign me in openstack-manuals if it's on docs.openstack;22:48
fungiannegentle: i can certainly shop it around our meeting tomorrow and start a ml thread to get some interest in doing the implementation work22:49
Sam-I-Amannegentle: this particular issue is a devref thing, but i see it come up now and then with other things22:49
annegentlefungi: sounds good, I'll join in and add it to the agenda for tomorrow. I've asked jhesketh since he knows alot about the swift logging blueprint; he's not sure he could take it on22:50
*** esker has quit IRC22:50
*** hdd has joined #openstack-infra22:50
fungiannegentle: thanks! and yeah, i'm sure jhesketh is already spread pretty thin22:50
gtmanfredSpamapS: awesome, just saw the prs today22:50
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Remove trusted-hosts from pip.conf  https://review.openstack.org/27363622:50
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Removed pypi.<region> mirror manifests  https://review.openstack.org/27351822:50
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351922:50
jeblairkrotscheck1: why the rebase on 273636?22:52
krotscheck1becayse distracted by baby22:52
krotscheck1gumme a few22:52
jeblairkrotscheck1: i'll fix it for you :)22:54
openstackgerritJames E. Blair proposed openstack-infra/system-config: Removed pypi.<region> mirror manifests  https://review.openstack.org/27351822:55
openstackgerritJames E. Blair proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351922:55
*** bpokorny_ has joined #openstack-infra22:56
*** kzaitsev_mb has quit IRC22:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove trusted-hosts from pip.conf  https://review.openstack.org/27363622:57
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove trusted-hosts from pip.conf  https://review.openstack.org/27363622:57
clarkbinfra-root doesn't appear to forward to anything22:57
jeblairthat's harder to fix than it shoud be :(22:57
clarkband I don't see creds for openstack-infra@openstack.org so I think infra-root is the one I want22:58
openstackgerritJames E. Blair proposed openstack-infra/system-config: Removed pypi.<region> mirror manifests  https://review.openstack.org/27351822:58
openstackgerritJames E. Blair proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351922:58
clarkbjeblair: do you know if the intent with infra-root was we would just access that mailbox directly rather than via forward/alias?22:58
krotscheck1jeblair: back22:59
jeblairclarkb: yep22:59
fungiclarkb: i think so, yes22:59
* krotscheck1 wishes that dependent patches were a little more ovious22:59
openstackgerritsebastian marcet proposed openstack-infra/openstackid: Fix on user idenfier contruction  https://review.openstack.org/27491722:59
clarkbcool I am just going to go ahead and use that address since it seems saner22:59
jeblairclarkb: yes22:59
*** bpokorny has quit IRC22:59
jeblairclarkb: note i think we also used jenkins@o.o for the second accounts22:59
jeblair(assuming the addr needs to be different)22:59
*** jsavak has quit IRC23:00
clarkbjeblair: ok can let them know that23:00
fungiokay, so our ansible inventory seems to be getting created by os-client-config. does that sound correct to anyone else?23:00
clarkband that is an alias for infra-root23:00
jeblairclarkb: yup23:01
*** weshay has quit IRC23:01
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351923:01
krotscheck1ok.23:01
fungithe only reference in our puppet i find to doing anything in that path other than deleting the cache is in our clouds.yaml template23:01
krotscheck1Let me just take all those individually and rebase them on master23:02
*** yuanying has joined #openstack-infra23:02
fungiso it seems to me that when we're running client calls, oscc is caching our nova list there in a format which we subsequently use as our ansible inventory file23:02
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351923:02
fungiit's not yet clear to me what routine process would be triggering that on the puppetmaster23:03
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Update bandersnatch documentation  https://review.openstack.org/27351923:03
fungithough mordred had previously indicated that blowing away the inventory cache would cause it to be regenerated23:03
jeblairfungi: i think the puppet runs23:03
*** dingyichen has joined #openstack-infra23:04
*** pradk has quit IRC23:04
*** regXboi has quit IRC23:05
*** hashar has quit IRC23:05
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Removed pypi.<region> mirror manifests  https://review.openstack.org/27351823:05
fungiwell, the wrapper script called from our cron jobs doesn't seem to do anything other than call ansible playbooks, so if it's self-healing at each run then presumably one of the playbooks is making a call to something which imports oscc23:06
krotscheck1jeblair: WIPing ^^23:06
krotscheck1Oh wait, it still has your -2.23:06
krotscheck1n/m23:06
jeblairfungi: ansible uses shade which uses occ23:07
nibalizerjeblair: fungi yes that23:07
krotscheck1Ok, I think that handles bandersnatch doc and the trusted host removal and the legacy manifest removal for later.23:08
krotscheck1Now, on to wheeling things23:08
fungiyeah, that's the path to it i'm finding. so presumably oscc has some built-in cache refresh age which would cause this to update periodically, but will create it anew if missing23:08
nibalizerfungi: what is the bug?23:08
*** annegentle has quit IRC23:08
clarkbresponse sent to osic23:08
jeblairfungi: i think it's ansible inventory that has the cache, and i think it's 24h23:08
funginibalizer: hpcloud pypi mirror became unreachable and our ansible runs are spending a couple hours waiting for it to respond before timing out23:09
fungiso anyway, i'm reasonably convinced that it really is just that it's been less than 24 hours since the change merged to tear that our of our host list, and that deleting the cache is the safest solution23:09
jeblairfungi: did you see the note from Shrews earlier about the refresh command?23:09
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added afs to wheel mirror slaves  https://review.openstack.org/27305423:09
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Add AFS and Admin keytab to release slave.  https://review.openstack.org/27353823:09
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added AFS keytab to wheel slave  https://review.openstack.org/27320623:09
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added conditional install statements for kernel header packages  https://review.openstack.org/27320523:09
openstackgerritMichael Krotscheck proposed openstack-infra/system-config: Added wheel hosting for AFS mirror slave  https://review.openstack.org/27097923:09
krotscheck1Ok, those are now rebased on master and not on afs-mirror things23:10
fungioh, sorry Shrews i was probably skimming23:10
* krotscheck1 looks at project-config things23:10
*** xyang1 has quit IRC23:10
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Check branch-override in ofagent job-template  https://review.openstack.org/27493023:10
jeblairi have to run now23:10
fungiShrews: what i should have asked is "refresh option where for the inventory?"23:10
fungijeblair: thanks!23:10
fungiShrews: like an on-demand thing, or you mean a config option to tune the refresh interval?23:11
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh  https://review.openstack.org/26711723:13
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Added wheel-release job  https://review.openstack.org/27354923:13
openstackgerritMichael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror  https://review.openstack.org/16492723:13
krotscheck1And that's that.23:13
*** piet has joined #openstack-infra23:14
openstackgerritMerged openstack-infra/openstackid: Fix on user idenfier contruction  https://review.openstack.org/27491723:14
*** armax has quit IRC23:15
*** david-lyle has quit IRC23:15
fungitoday i learned ... mumble mumble ... ansible openstackclient shade oscc inventory cache ... mumble mumble23:16
clarkbharlowja: I never heard back on your end about talking about cloud resources. Just ping me on irc or via email if you want ot set something up23:16
Shrewsfungi: i have no idea how you're using the inventory. are you going through shade directly or using the inventory module provided by ansible?23:16
harlowjaclarkb ya, i think management here wanted to have a internal chat first, then pull u in :)23:16
harlowjau or someone in23:16
Shrewsfungi: module has a --refresh option, shade has a 'refresh' parameter23:16
harlowjasaid internal chat/meeting hasn't happened yet23:16
clarkbharlowja: gotcha, well just let me/us know if/when you get to that point23:17
harlowjakk23:17
fungiShrews: this is all under ansible, was just trying to narrow down the symptoms we were seeing to be sure we weren't failing to refresh the inventory cache23:17
Shrewsfungi: also, i *think* the cache is just a file? could possibly delete it, but ymmv23:17
clarkbdogpile uses a file by default23:17
ianwgreghaynes: ok, looks like upstream issue -> https://bugzilla.redhat.com/show_bug.cgi?id=1303660 .  probably a botched fix23:17
openstackbugzilla.redhat.com bug 1303660 in python-rsa "Broken Dependencies for python2-rsa-3.3-2.el6.noarch" [Unspecified,New] - Assigned to yohangraterol9223:17
fungiand yes, it's a cache, and yes i did just delete it a minute ago (well, moved it to my homedir so i can diff the auto-generated one against it shortly)23:17
fungier, i meant yes it's a file23:18
Shrewsfungi: https://github.com/ansible/ansible/blob/938c26d9524f0eecff03abc2678e6ccc5b36b034/contrib/inventory/openstack.py#L184-L18523:19
*** annegentle has joined #openstack-infra23:20
krotscheck1Oh look!23:20
krotscheck1It's a bunch of patches!23:20
krotscheck1Passing gate!23:20
krotscheck1With a core +2 already! https://review.openstack.org/#/q/status:open+project:openstack-infra/system-config+branch:master+topic:wheel_mirror23:21
krotscheck1What does it meaaaan! #doublerainbow23:21
*** sigmavirus24 is now known as sigmavirus24_awa23:22
*** rocky_g has quit IRC23:25
*** rbrndt has quit IRC23:26
*** vivekd has quit IRC23:27
krotscheck1Hrm.23:29
*** armax has joined #openstack-infra23:29
krotscheck1jeblair: Sorry, was rushing through fixes. I'll look at the wheel-release job in the morning.23:30
krotscheck1Or later tonight if the mucnhkin takes a nap23:30
greghaynesianw: awesome23:31
fungidoes this munchkin represent the lollipop guild?23:31
greghaynesianw: so we find useful things!23:31
krotscheck1fungi: Not yet, no sugar until he's at least 3.23:32
fungigot it, so lullaby league23:32
*** mriedem_away is now known as mriedem23:32
*** rlandy has quit IRC23:34
openstackgerritIan Wienand proposed openstack/diskimage-builder: Skip centos functional testing  https://review.openstack.org/27493723:35
*** gordc has quit IRC23:36
*** gildub has joined #openstack-infra23:36
fungiShrews: thanks! probably something that would take a bit of hackery to employ in situations where a host was removed from the inventory and then taken offline between typical cache refresh intervals. easier to just document blowing away the cache in these situations i guess23:36
*** krotscheck1 has quit IRC23:36
*** jamielennox is now known as jamielennox|away23:36
fungirather than trying to detect the condition and remedy automagically23:36
greghaynesianw: given that it is a real fail we found, I am less crazy about disabling that test23:37
*** salv-orl_ has joined #openstack-infra23:37
greghaynesianw: I am more concerned with the sporadic mirror fails, which I think is mostly a fedora thing?23:37
*** dtardivel has quit IRC23:38
*** dimtruck is now known as zz_dimtruck23:38
*** dalgaaf has quit IRC23:38
openstackgerritIan Wienand proposed openstack/diskimage-builder: [DNM] Revert "Skip centos functional testing"  https://review.openstack.org/27493823:38
*** kragniz_ is now known as kragniz23:38
ianwgreghaynes: so this failure is a hard-blocker until the upstream package is fixed.23:38
ianwsee how the skip goes ... i can also put in a workaround23:40
ianwbut really, i think i'll switch the functional testing over to fedora-minimal & centos-minimal builds23:40
*** salv-orlando has quit IRC23:40
ianwrealistically, that is all i'm maintaining ... i doubt anyone cares about the cloud-image based builds23:41
*** markvoelker has quit IRC23:41
*** MarkAtwood has joined #openstack-infra23:42
greghaynesianw: heh, fair enough23:43
openstackgerritEric Brown proposed openstack/requirements: Bump Bandit from 0.13.2 to 0.17.3  https://review.openstack.org/27493923:44
fungion a related note, i need to add some more bindep-fallback experimental jobs for different node types if we're close to having more dib-only types on rackspace23:45
fungithe one for ubuntu-trusty is passing both for bindep changes and project-config changes, and exercising the bindep fallback path in the job script at the same time23:46
madornhi, does anyone know the status of enabling +2 reviews on openstack-dev/sandbox @ review.openstack.org? it has been broken since the Dec upgrade.23:46
ianwfungi: yeah, centos dib for rax is on my todo list, before being preempted (as usual).  i'm not sure about it, so i will add it as a separate node & experimental job first i think23:47
fungimadorn: zaro found and fixed a bug in gerrit upstream, which needs to be backported to 2.11 (if it's merged yet)23:47
fungithough i think there was a related workaround in jeepyb which may fix it for us sooner23:48
madornfungi: okay cool thanks for the info23:48
fungiif that's merged, then it should just be a matter of figuring out how to clean up the mess with the broken groups23:48
fungiwhich probably requires gerrit downtime23:48
fungiso will need to be scheduled once identified and tried out on review-dev23:49
madornfungi: cool, let me know if i can help in anyway, thanks.23:49
fungimadorn: hopefully zaro sees this and can chime in with a more detailed status on the problem. i don't know if he's determined yet what the safest cleanup steps are23:50
*** EricGonczer_ has joined #openstack-infra23:51
*** amotoki has joined #openstack-infra23:51
fungii'm guessing it's something like manually editing the groups files of all affected repos and then pushing those updates back into the gerrit meta/config ref23:51
fungiso maybe that can be done hot23:52
*** yamahata has quit IRC23:53
*** Swami has quit IRC23:55
*** yamahata has joined #openstack-infra23:56
*** bpokorny_ has quit IRC23:57
madornfungi: cool, do you by any chance know the upstream gerrit bug #23:57
*** bpokorny has joined #openstack-infra23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!