Tuesday, 2016-10-18

*** amotoki has quit IRC00:00
*** markvoelker has joined #openstack-infra00:01
*** amitgandhinz has quit IRC00:01
*** esikachev has quit IRC00:02
*** hogepodge has quit IRC00:02
mordredjeblair: I re-run now00:03
mordredrunning remote_puppet_else00:03
jeblairi've seen cacti write out to rrd files now00:04
clarkbjeblair: and I can find the username/password for that on the cacti host? (not seeing them in hiera000:04
jeblairi think the rebuild may have helped with that...00:04
jeblairclarkb: password file00:04
clarkbah00:04
jeblairi'm not sure about the parallelization00:04
jeblairthere are a bunch of warnings now about missing oids from compute hosts00:04
jeblairmaybe their networks changed?00:05
jeblair(interfaces, etc)00:05
*** weshay_bbiab is now known as weshay00:05
rcarrillocruzyep00:05
rcarrillocruzhttp://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=30000:05
rcarrillocruzgraphs now showing00:05
jeblairthe create_graphs script doesn't delete anything00:05
*** amotoki has joined #openstack-infra00:05
jeblairso i think we may need to delete info for those hosts and start over00:06
*** dimtruck is now known as zz_dimtruck00:06
*** markvoelker has quit IRC00:06
jeblairi'm disabling compute025 in cacti00:08
*** hogepodge has joined #openstack-infra00:08
jeblairand compute01600:09
*** david-lyle has quit IRC00:09
jeblairand compute00800:09
jeblairokay, the oid errors i'm seeing aren't related to parallelization00:10
jeblairi think we have a problem with cacti and infra-cloud00:11
jeblairwe run the create_graphs script in a cron00:11
jeblairevery minute, apparently.00:11
*** xarses has joined #openstack-infra00:11
jeblairsorry, every hour00:12
*** sabari has quit IRC00:12
jeblairand it scans the network interfaces on each host and adds them00:12
jeblairi think that the tap devices, etc, that are created on the compute nodes all show up00:12
jeblairso every hour, we add a new batch of randomly named interfaces00:12
jeblairwhich is why compute009 has 1262 data sources00:13
clarkboh interesting00:13
mordredoh wow. yeah.00:13
rcarrillocruzcan we blacklist the tap interfaces collecting on a glob?00:13
rcarrillocruzlike, don't collect tap on compute*.vanilla.ic.openstack.org or somethjing00:14
rcarrillocruzin cacti that is00:14
rcarrillocruz?00:14
jeblairso i think this probably corresponds to us *using* infracloud for real (which is why the errors didn't line up with us adding ic to cacti or anything like that)00:14
mordredyah. because those only happen on usage00:14
mordredI mean, in a long-term view, collecting data about usage on those interfaces is interesting - although likely only in aggregate00:14
mordredbut shorter-term not collecting info on those interfaces seems better00:15
clarkbmordred: and if not in aggregate need better mapping to instances00:15
mordredyah00:15
mordredwhich is several steps away from useful today00:15
*** jamesde__ has quit IRC00:17
jeblairrcarrillocruz: looking into that00:17
clarkbjeblair: rcarrillocruz and if not based on name glob maybe we can base it on the hardware address type like glean does?00:18
jeblairrcarrillocruz: i think we would have to create a script query: http://www.cacti.net/downloads/docs/html/script_query_xml.html00:18
jeblaircurrently the thing that gets the list of interfaces is an 'snmp query': http://cacti.openstack.org/cacti/data_queries.php?action=edit&id=100:19
jeblairhttp://www.cacti.net/downloads/docs/html/snmp_query_xml.html00:19
jeblairand skimming the docs for that, i don't see an easy way to filter00:19
*** hogepodge has quit IRC00:20
*** hogepodge has joined #openstack-infra00:21
jeblairrcarrillocruz: oh... or we could handle it ourselves in create_graphs.sh00:21
jeblairwe actually do that for disks now00:21
*** david-lyle has joined #openstack-infra00:21
rcarrillocruzlet me see the script00:21
*** gyee has quit IRC00:25
*** kzaitsev_mb has quit IRC00:27
rcarrillocruzhttp://paste.openstack.org/show/586068/00:28
rcarrillocruzso what's stuff are we insterestd in00:28
*** yolanda has quit IRC00:28
rcarrillocruzwe grep -v the taps and brq ?00:29
rcarrillocruzjust the taps?00:29
*** mtanino__ has joined #openstack-infra00:29
*** yolanda has joined #openstack-infra00:29
*** david-lyle has quit IRC00:29
jeblairyeah, that sounds like a good start00:30
*** mtanino has quit IRC00:30
jeblairrcarrillocruz: so then we'd change "ifOperStatus" "Up"  to  "ifName" and $iface -- and make $iface a loop variable like $disk in the thing below00:30
*** amotoki has quit IRC00:30
*** david-lyle has joined #openstack-infra00:31
rcarrillocruzgotcha00:31
jeblairlike http://paste.openstack.org/show/586070/00:32
*** asselin__ has quit IRC00:33
jeblairi'm disabling poller logging entirely and bumping the number of pollers back up to 400:33
jeblair(the logging will flood with warnings for all the transient interfaces)00:33
jeblairrcarrillocruz: do you want to update the create_graphs script or should i?00:34
rcarrillocruzi'm poking at it, i'll ask you to review00:34
*** david-lyle has quit IRC00:34
jeblairk00:34
*** david-lyle has joined #openstack-infra00:35
rcarrillocruzhttp://paste.openstack.org/show/586071/00:37
rcarrillocruzhow does that look like00:37
jeblairrcarrillocruz: looks good but you want to add "| grep -v brq" to that too?00:38
rcarrillocruzah k, understood just the tap00:38
rcarrillocruz++00:38
rcarrillocruzi'll push for review then00:38
rcarrillocruzwith brq00:38
jeblaircool00:38
jeblairrcarrillocruz: i just tried that out too, looks like it should work00:39
*** amotoki has joined #openstack-infra00:40
*** sabari has joined #openstack-infra00:41
jeblairrcarrillocruz: so we merge that, then remove all the extra data sources.00:41
jeblairi think that will actually be easy to do in the webui00:41
jeblair(data sources and graphs)00:41
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Filter out brq and tap devices from net interfaces monitoring  https://review.openstack.org/38773800:42
rcarrillocruz++00:42
jeblairaprvd00:42
jeblairokay, i manually installed that on cacti00:43
rcarrillocruzsweet :-)00:44
jeblairi will delete the extra graphs now00:44
rcarrillocruzso out of curiosity, for the admin ui, the creds are on the php files i guess right?00:44
*** amotoki has quit IRC00:44
jeblairrcarrillocruz: they are in the passwords.gpg file on puppetmaster00:45
jeblair(the account in cacti is in mysql db)00:45
rcarrillocruzah k00:45
jeblairokay, cleared out extra graphs, and re-set poller log level to lowv00:47
*** david-lyle has quit IRC00:47
*** david-lyle has joined #openstack-infra00:48
*** amotoki has joined #openstack-infra00:48
jeblairi think i just missed the 0045 run with the log level change00:48
jeblairmay have to wait until 0055 to see if it worked00:49
*** sdake_ has joined #openstack-infra00:51
*** EricGonczer_ has quit IRC00:51
*** sdake_ has joined #openstack-infra00:51
*** david-lyle has quit IRC00:51
jeblairoh, still have more data sources to delete.  working on it.00:52
*** david-lyle has joined #openstack-infra00:52
*** sdake has quit IRC00:53
jeblairrcarrillocruz, clarkb, mordred: are you still around, or are any other infra-root around?00:53
*** claudiub has quit IRC00:53
rcarrillocruzi am00:53
jeblairi have about 100 pages to click through in cacti to do this; would go faster with some help :)00:54
rcarrillocruzheh, i need to know how to get decrypt passwords.gpg00:54
openstackgerritMerged openstack-infra/system-config: Filter out brq and tap devices from net interfaces monitoring  https://review.openstack.org/38773800:54
rcarrillocruzthat or some other core who has the user/pass freshh00:55
rcarrillocruzclarkb: around?00:55
clarkbsorry I have disappeared for family thing wont be able to help now00:56
rcarrillocruzno probs, mordred ^ ?00:57
*** amitgandhinz has joined #openstack-infra00:58
*** david-lyle has quit IRC00:59
*** david-lyle has joined #openstack-infra01:00
*** jamielennox is now known as jamielennox|away01:01
*** lifeless has quit IRC01:02
*** lifeless has joined #openstack-infra01:02
jeblairhalfway done01:02
* anteaya cheers on jeblair's mouse fingers01:04
jeblairi don't have a mouse01:04
pleia2I can pitch in now, where are we jeblair?01:04
anteayaah, you said clicking and I heard mouse01:04
anteayaclicking fingers01:05
*** david-lyle has quit IRC01:05
*** tqtran has quit IRC01:05
*** david-lyle has joined #openstack-infra01:06
jeblairpleia2: i'm close enough to done that it may not be worth jumping in at this point01:06
pleia2ok, sorry01:07
jeblairpleia2: but maybe you could share the password file password with rcarrillocruz?01:07
pleia2sure01:07
rcarrillocruz++01:07
*** rhallisey has quit IRC01:07
pleia2rcarrillocruz: can you sudo su - on puppetmaster so I can give a quick tour?01:07
rcarrillocruzsure01:07
rcarrillocruzin01:07
*** david-lyle has quit IRC01:07
pleia2while I figure out how to quit emacs01:07
rcarrillocruzhaha01:08
rcarrillocruzyeah, i'm  a vim user myself :/01:08
rcarrillocruzctrl X or something like that01:08
pleia2I kid anyway ;)01:08
*** jamielennox|away is now known as jamielennox01:08
jeblairshe knows i can't type a rejoinder :)01:08
pleia2rcarrillocruz: screen -x01:09
*** jkilpatr has quit IRC01:09
rcarrillocruzo-k01:10
pleia2rcarrillocruz is all sorted :)01:10
*** Apoorva_ has joined #openstack-infra01:10
rcarrillocruzi know have it on a post it on my laptop01:11
jeblairyay!01:11
* rcarrillocruz giggles01:11
jeblairrcarrillocruz:  ;)01:11
rcarrillocruz;-)01:11
pleia2lol01:11
jeblairi have just finished deleting all the graphs01:11
rcarrillocruzyeah thanks jeblair , sorry for the grunt work01:11
rcarrillocruzand we haz cacti!01:11
rcarrillocruz\o/01:11
*** yamahata has quit IRC01:12
jeblair 10/18/2016 01:11:40 AM - SYSTEM STATS: Time:98.8470 Method:cmd.php Processes:4 Threads:N/A Hosts:182 HostsPerProcess:46 DataSources:23559 RRDsProcessed:856401:12
*** knangia has quit IRC01:12
jeblairthat last number is no longer 0 :)01:12
rcarrillocruzthat looks so much better than 0 hehe01:12
*** david-lyle has joined #openstack-infra01:13
rcarrillocruzthanks for all the help, appreciated01:13
rcarrillocruzcrashing now (3:13AM)01:13
jeblairrcarrillocruz: thank you!01:13
rcarrillocruztty tomorrow folks01:13
rcarrillocruz++01:13
*** david-lyle has quit IRC01:14
*** Apoorva has quit IRC01:14
amrithhi infra ... would someone please provide feedback on https://review.openstack.org/#/c/387386/. It is part of getting rid of the trove-integration repository and is currently on the critical path. Thanks01:14
jeblairi'm going to clean up the tree01:14
*** david-lyle has joined #openstack-infra01:15
*** Apoorva_ has quit IRC01:15
*** hogepodge has quit IRC01:16
*** yolanda has quit IRC01:17
ianychoiHi pleia2, I think your timezone is around dinner time! How about discussing translation checksite on next infra team IRC meeting (tomorrow 1900 UTC)? I think Frank from I18n team will be able to join to the meeting.01:17
pleia2ianychoi: yeah, let's go ahead and add it to the agenda01:18
*** shu-mutou-AWAY is now known as shu-mutou01:18
pleia2I'll do my best to get up to speed by meeting time01:18
*** david-lyle has quit IRC01:19
ianychoipleia2, yep I will add to wiki agenda thank you!01:19
pleia2ianychoi: can you make sure you include the link to frank's email?01:19
pleia2I know it went to the infra list, just want to make sure it's called out01:19
ianychoipleia2, sure! I include the link to infra meeting wiki01:19
pleia2perfect, thanks01:19
*** david-lyle has joined #openstack-infra01:20
*** baoli has joined #openstack-infra01:21
ianychoiadded :) see you tomorrow!01:21
pleia2great, see you then01:21
ianychoi:)01:21
*** david-lyle has quit IRC01:21
*** pcrews has quit IRC01:22
*** john-davidge has joined #openstack-infra01:23
*** david-lyle has joined #openstack-infra01:23
*** yanyanhu has joined #openstack-infra01:23
*** yolanda has joined #openstack-infra01:23
*** david-lyle has quit IRC01:24
*** david-lyle has joined #openstack-infra01:25
*** john-davidge has quit IRC01:27
*** kaisers_ has joined #openstack-infra01:27
*** jamesden_ has joined #openstack-infra01:28
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Refactor run-tox-with-oslo-master.sh  https://review.openstack.org/38768101:28
*** yamahata has joined #openstack-infra01:30
*** amitgandhinz has quit IRC01:32
*** kaisers_ has quit IRC01:32
*** david-lyle has quit IRC01:32
*** pcrews has joined #openstack-infra01:33
*** pahuang has quit IRC01:33
*** david-lyle has joined #openstack-infra01:34
*** thorst_ has joined #openstack-infra01:34
*** hogepodge has joined #openstack-infra01:35
*** david-lyle has quit IRC01:37
*** yee3796 has joined #openstack-infra01:38
*** david-lyle has joined #openstack-infra01:38
*** david-lyle has quit IRC01:38
*** yee379 has quit IRC01:39
*** jamesden_ has quit IRC01:40
*** jamesden_ has joined #openstack-infra01:41
*** NobodyCam has quit IRC01:42
*** tdurakov has quit IRC01:42
*** tdurakov has joined #openstack-infra01:42
*** NobodyCam has joined #openstack-infra01:42
*** david-lyle has joined #openstack-infra01:44
*** david-lyle has quit IRC01:44
*** david-lyle has joined #openstack-infra01:45
*** pahuang has joined #openstack-infra01:45
*** vdrok has quit IRC01:45
*** david-lyle has quit IRC01:46
*** jamesden_ has quit IRC01:46
*** gildub_ has joined #openstack-infra01:46
*** david-lyle has joined #openstack-infra01:46
*** david-lyle has quit IRC01:47
*** vdrok has joined #openstack-infra01:48
*** gildub has quit IRC01:49
*** gongysh has joined #openstack-infra01:50
*** evgenyl has quit IRC01:51
*** larainema has quit IRC01:51
*** evgenyl has joined #openstack-infra01:52
*** csomerville has quit IRC01:52
*** larainema has joined #openstack-infra01:53
*** david-lyle has joined #openstack-infra01:56
*** david-lyle has quit IRC01:57
*** david-lyle has joined #openstack-infra01:58
*** mtanino has joined #openstack-infra01:59
*** baoli has quit IRC01:59
*** esikachev has joined #openstack-infra01:59
*** david-lyle has quit IRC02:00
*** mtanino__ has quit IRC02:00
*** jamesden_ has joined #openstack-infra02:01
*** david-lyle has joined #openstack-infra02:01
*** david-lyle has quit IRC02:02
*** david-lyle has joined #openstack-infra02:02
*** markvoelker has joined #openstack-infra02:03
*** david-lyle has quit IRC02:03
*** david-lyle has joined #openstack-infra02:04
*** esikachev has quit IRC02:04
*** david-lyle has quit IRC02:04
*** mtanino has quit IRC02:04
*** david-lyle has joined #openstack-infra02:05
*** david-lyle has quit IRC02:05
openstackgerritMerged openstack-infra/devstack-gate: Reorganize PROJECTS list to show repos to remove  https://review.openstack.org/38761402:05
*** cody-somerville has joined #openstack-infra02:06
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775002:06
*** david-lyle has joined #openstack-infra02:07
*** markvoelker has quit IRC02:08
*** david-lyle has quit IRC02:08
*** cody-somerville has quit IRC02:09
*** cody-somerville has joined #openstack-infra02:10
*** david-lyle has joined #openstack-infra02:10
*** david-lyle has quit IRC02:11
*** david-lyle has joined #openstack-infra02:12
*** jamesde__ has joined #openstack-infra02:12
*** david-lyle has quit IRC02:14
*** jamesden_ has quit IRC02:14
*** tuanluong has joined #openstack-infra02:14
*** Swami__ has quit IRC02:15
*** david-lyle has joined #openstack-infra02:17
*** david-lyle has quit IRC02:18
*** david-lyle has joined #openstack-infra02:18
*** david-lyle has quit IRC02:19
*** david-lyle has joined #openstack-infra02:20
*** gouthamr has quit IRC02:21
*** netsin has quit IRC02:21
*** david-lyle has quit IRC02:21
*** zz_dimtruck is now known as dimtruck02:23
*** baoli has joined #openstack-infra02:25
*** hongbin_ has joined #openstack-infra02:26
*** david-lyle has joined #openstack-infra02:26
*** dave-mccowan has quit IRC02:26
*** baoli_ has joined #openstack-infra02:28
*** amitgandhinz has joined #openstack-infra02:28
*** yuanying has quit IRC02:29
*** tiswanso has joined #openstack-infra02:30
*** baoli has quit IRC02:31
*** baoli_ has quit IRC02:39
*** armax has quit IRC02:39
*** jamesde__ has quit IRC02:40
*** hogepodge has quit IRC02:40
*** hongbin_ has quit IRC02:41
*** hongbin_ has joined #openstack-infra02:41
*** jamesden_ has joined #openstack-infra02:42
*** armax has joined #openstack-infra02:42
*** Rockyg has quit IRC02:46
*** jamesden_ has quit IRC02:50
*** esikachev has joined #openstack-infra03:00
*** jamielennox is now known as jamielennox|away03:01
*** netsin has joined #openstack-infra03:02
*** amitgandhinz has quit IRC03:02
*** thorst_ has quit IRC03:04
*** hogepodge has joined #openstack-infra03:04
*** esikachev has quit IRC03:04
*** thorst_ has joined #openstack-infra03:05
*** thorst_ has quit IRC03:05
*** thorst_ has joined #openstack-infra03:06
*** thorst_ has quit IRC03:13
*** jbryce has quit IRC03:16
*** jbryce has joined #openstack-infra03:16
*** kaisers_ has joined #openstack-infra03:16
*** coolsvap has quit IRC03:20
*** hongbin_ has quit IRC03:20
*** vikrant has joined #openstack-infra03:21
*** kaisers_ has quit IRC03:21
*** david-lyle has quit IRC03:23
*** david-lyle has joined #openstack-infra03:24
*** david-lyle has quit IRC03:25
*** david-lyle has joined #openstack-infra03:26
*** Sukhdev has joined #openstack-infra03:26
openstackgerritIan Wienand proposed openstack-infra/project-config: Fix unbound rc.local generation  https://review.openstack.org/38715003:27
ianwsight, i wish i didn't know so much about rc.local, but there you go03:27
*** david-lyle has quit IRC03:27
*** david-lyle has joined #openstack-infra03:28
*** david-lyle has quit IRC03:29
*** david-lyle has joined #openstack-infra03:30
*** david-lyle has quit IRC03:33
*** jamielennox|away is now known as jamielennox03:33
*** david-lyle has joined #openstack-infra03:34
*** tiswanso has quit IRC03:35
*** links has joined #openstack-infra03:35
*** armax has quit IRC03:36
*** david-lyle has quit IRC03:36
*** amotoki_ has joined #openstack-infra03:40
*** amotoki has quit IRC03:43
*** david-lyle has joined #openstack-infra03:46
*** david-lyle has quit IRC03:46
openstackgerritIan Wienand proposed openstack-infra/devstack-gate: Remove dib-utils & diskimage-builder from PROJECTS  https://review.openstack.org/34581803:48
*** yuanying has joined #openstack-infra03:49
*** john-davidge has joined #openstack-infra03:49
*** david-lyle has joined #openstack-infra03:50
*** david-lyle has quit IRC03:51
*** david-lyle has joined #openstack-infra03:53
*** e0ne has joined #openstack-infra03:54
*** david-lyle has quit IRC03:54
*** john-davidge has quit IRC03:54
*** david-lyle has joined #openstack-infra03:55
*** david-lyle has quit IRC03:55
*** tqtran has joined #openstack-infra03:57
*** david-lyle has joined #openstack-infra03:57
*** david-lyle has quit IRC03:57
*** yamahata has quit IRC03:58
*** e0ne has quit IRC03:58
*** david-lyle has joined #openstack-infra03:58
*** david-lyle has quit IRC03:59
*** amitgandhinz has joined #openstack-infra03:59
*** psachin` has joined #openstack-infra03:59
*** e0ne has joined #openstack-infra04:00
*** david-lyle has joined #openstack-infra04:00
*** david-lyle has quit IRC04:01
*** esikachev has joined #openstack-infra04:01
*** david-lyle has joined #openstack-infra04:01
*** sabari has quit IRC04:02
*** david-lyle has quit IRC04:04
*** david-lyle has joined #openstack-infra04:04
*** markvoelker has joined #openstack-infra04:04
*** esikachev has quit IRC04:05
*** david-lyle has quit IRC04:05
*** david-lyle has joined #openstack-infra04:06
*** sabari has joined #openstack-infra04:07
*** david-lyle has quit IRC04:08
*** markvoelker has quit IRC04:09
*** david-lyle has joined #openstack-infra04:10
*** david-lyle has quit IRC04:11
*** thorst_ has joined #openstack-infra04:11
*** david-lyle has joined #openstack-infra04:12
*** david-lyle has quit IRC04:13
*** david-lyle has joined #openstack-infra04:14
openstackgerritJames E. Blair proposed openstack-infra/storyboard: Update story.updated_at when tags are updated  https://review.openstack.org/38778304:15
*** david-lyle has quit IRC04:15
openstackgerritJames E. Blair proposed openstack-infra/storyboard: Update story.updated_at when comments added  https://review.openstack.org/38778404:15
*** david-lyle has joined #openstack-infra04:15
*** david-lyle has quit IRC04:16
*** david-lyle has joined #openstack-infra04:17
*** david-lyle has quit IRC04:18
*** thorst_ has quit IRC04:18
*** david-lyle has joined #openstack-infra04:19
*** david-lyle has quit IRC04:20
*** cshastri has joined #openstack-infra04:21
*** david-lyle has joined #openstack-infra04:21
*** amotoki_ has quit IRC04:28
*** sdake_ has quit IRC04:30
*** sdake has joined #openstack-infra04:33
*** amitgandhinz has quit IRC04:33
openstackgerritMerged openstack-infra/release-tools: add a --cycle-based option to list-repos  https://review.openstack.org/38764904:41
*** markvoelker has joined #openstack-infra04:41
*** markvoelker_ has joined #openstack-infra04:42
*** markvoelker has quit IRC04:45
*** mdrabe has quit IRC04:47
*** kzaitsev_mb has joined #openstack-infra04:49
*** yuanying has quit IRC04:54
*** rwsu has joined #openstack-infra04:56
*** markvoelker_ has quit IRC04:58
*** yuanying has joined #openstack-infra05:00
*** esikachev has joined #openstack-infra05:01
*** yuanying has quit IRC05:02
*** kaisers_ has joined #openstack-infra05:05
*** esikachev has quit IRC05:06
*** sc68cal has quit IRC05:07
*** kaisers_ has quit IRC05:10
*** sdake_ has joined #openstack-infra05:11
*** markvoelker has joined #openstack-infra05:13
*** kzaitsev_mb has quit IRC05:13
*** adriant has quit IRC05:14
*** markvoelker_ has joined #openstack-infra05:14
*** sdake has quit IRC05:14
*** thorst_ has joined #openstack-infra05:17
*** markvoelker has quit IRC05:18
*** jaosorior has joined #openstack-infra05:18
*** yamamoto has quit IRC05:20
*** thorst_ has quit IRC05:23
*** hurgleburgler has quit IRC05:23
*** amitgandhinz has joined #openstack-infra05:30
*** tqtran has quit IRC05:32
*** amotoki has joined #openstack-infra05:36
*** eliqiao has joined #openstack-infra05:43
*** markvoelker_ has quit IRC05:45
*** chlong has quit IRC05:47
*** amotoki has quit IRC05:52
*** dimtruck is now known as zz_dimtruck05:56
*** nadya has joined #openstack-infra05:56
*** AJaeger has quit IRC05:56
*** amotoki has joined #openstack-infra05:57
*** yamamoto has joined #openstack-infra05:57
openstackgerritThomas Bechtold proposed openstack-infra/jenkins-job-builder: Fix timeout wrapper version detection  https://review.openstack.org/38779905:58
*** nadya has quit IRC06:00
*** AJaeger has joined #openstack-infra06:00
*** rcernin has joined #openstack-infra06:01
*** yuanying has joined #openstack-infra06:01
*** links has quit IRC06:01
*** chlong has joined #openstack-infra06:01
*** esikachev has joined #openstack-infra06:02
*** ihrachys has joined #openstack-infra06:03
*** ccamacho has left #openstack-infra06:03
*** amitgandhinz has quit IRC06:04
*** amotoki has quit IRC06:04
*** amotoki has joined #openstack-infra06:05
*** AJaeger has quit IRC06:06
*** AJaeger has joined #openstack-infra06:06
*** sc68cal has joined #openstack-infra06:06
*** esikachev has quit IRC06:08
*** kaisers_ has joined #openstack-infra06:08
*** ihrachys has quit IRC06:10
*** vsaienko has joined #openstack-infra06:12
*** jtomasek_ has joined #openstack-infra06:13
*** john-davidge has joined #openstack-infra06:17
*** vsaienko has quit IRC06:17
*** pcaruana has joined #openstack-infra06:18
*** isaacb has joined #openstack-infra06:18
*** nadya has joined #openstack-infra06:20
*** nadya has quit IRC06:21
*** john-davidge has quit IRC06:21
*** esikachev has joined #openstack-infra06:22
*** aeng is now known as aeng_afk06:22
*** thorst_ has joined #openstack-infra06:23
*** chandankumar has joined #openstack-infra06:24
*** esikachev has quit IRC06:27
*** vsaienko has joined #openstack-infra06:27
*** gongysh has quit IRC06:27
*** Sukhdev has quit IRC06:27
*** thorst_ has quit IRC06:28
*** claudiub has joined #openstack-infra06:29
*** hogepodge has quit IRC06:30
*** claudiub|2 has joined #openstack-infra06:31
*** jtomasek_ has quit IRC06:32
*** claudiub has quit IRC06:34
*** nadya has joined #openstack-infra06:38
*** e0ne has quit IRC06:42
*** sdake_ is now known as sdake06:44
openstackgerritThomas Bechtold proposed openstack-infra/jenkins-job-builder: Fix timeout wrapper version detection  https://review.openstack.org/38779906:44
*** ihrachys has joined #openstack-infra06:45
*** andreas_s has joined #openstack-infra06:48
*** nherciu has joined #openstack-infra06:54
*** nadya has quit IRC07:00
*** amitgandhinz has joined #openstack-infra07:00
*** ihrachys has quit IRC07:02
*** yamahata has joined #openstack-infra07:03
*** tesseract has joined #openstack-infra07:03
*** tesseract is now known as Guest8585507:03
*** coolsvap has joined #openstack-infra07:04
*** ihrachys has joined #openstack-infra07:04
*** ccamacho has joined #openstack-infra07:05
*** camunoz has quit IRC07:07
*** shardy has joined #openstack-infra07:08
*** sree has joined #openstack-infra07:08
*** sree has quit IRC07:08
*** sree has joined #openstack-infra07:08
*** amotoki has quit IRC07:09
*** amotoki has joined #openstack-infra07:10
*** matrohon has joined #openstack-infra07:15
yolandagood morning07:15
*** odyssey4me has quit IRC07:16
*** amoralej|off is now known as amoralej07:16
*** clayton has quit IRC07:16
*** jmorgan1 has quit IRC07:17
*** jmorgan1 has joined #openstack-infra07:17
*** clayg has quit IRC07:18
*** amrith has quit IRC07:18
*** odyssey4me has joined #openstack-infra07:18
*** amrith has joined #openstack-infra07:19
*** clayton has joined #openstack-infra07:19
*** clayg has joined #openstack-infra07:19
*** dtardivel has joined #openstack-infra07:20
*** shardy has quit IRC07:22
*** tkelsey has joined #openstack-infra07:22
openstackgerritMerged openstack-infra/project-config: nova-lxd: Allow regex to blacklist tempest tests  https://review.openstack.org/38710007:23
*** vsaienko has quit IRC07:24
openstackgerritMerged openstack-infra/project-config: Enable py35 voting on passing keystone projects  https://review.openstack.org/38767007:24
*** thorst_ has joined #openstack-infra07:26
openstackgerritMerged openstack-infra/project-config: Change pep8 to voting for rsc  https://review.openstack.org/38761807:27
*** yaume has joined #openstack-infra07:28
*** yaume is now known as yaume[away]07:28
*** jordanP has joined #openstack-infra07:28
*** jordanP has quit IRC07:28
*** drifterza has joined #openstack-infra07:30
*** ralonsoh has joined #openstack-infra07:32
*** Naeil has joined #openstack-infra07:33
*** thorst_ has quit IRC07:33
*** amitgandhinz has quit IRC07:33
openstackgerritMerged openstack-infra/project-config: fix panko initialisation  https://review.openstack.org/38762007:34
openstackgerritMerged openstack-infra/project-config: Templatize freezer dsvm gate job  https://review.openstack.org/38609407:35
openstackgerritMerged openstack-infra/project-config: Add os-xenapi project.  https://review.openstack.org/38652807:35
openstackgerritMerged openstack-infra/project-config: gerritbot: Update repos and branches for charms  https://review.openstack.org/38729107:35
openstackgerritMerged openstack-infra/project-config: Move barbican-postgres to non-voting gate  https://review.openstack.org/38544807:37
*** vsaienko has joined #openstack-infra07:39
*** jpich has joined #openstack-infra07:39
*** hogepodge has joined #openstack-infra07:40
*** ihrachys has quit IRC07:41
AJaegeryolanda: could you review these later as well, please? https://review.openstack.org/#/c/386613/ https://review.openstack.org/386521 https://review.openstack.org/#/c/385863/07:42
yolandahi AJaeger , i'll do07:43
*** HeOS has quit IRC07:44
AJaegerthanks07:44
*** HeOS has joined #openstack-infra07:45
*** jpena|off is now known as jpena07:45
*** yaume[away] is now known as yaume07:46
*** yaume is now known as yaume[away]07:46
openstackgerritMerged openstack-infra/project-config: Add gerritbot and accessbot to openstack-interop  https://review.openstack.org/38578907:46
openstackgerritMerged openstack-infra/project-config: Link puppet-tripleo gerrit to tripleo launchpad  https://review.openstack.org/38266307:46
*** mrunge has quit IRC07:50
*** ihrachys has joined #openstack-infra07:51
*** mrunge has joined #openstack-infra07:53
*** yaume[away] is now known as yaume07:54
*** ihrachys has quit IRC07:56
*** igormarnat has quit IRC07:56
*** asilenkov has quit IRC07:56
*** dteselkin has quit IRC07:57
*** csomerville has joined #openstack-infra07:58
*** cody-somerville has quit IRC07:58
*** pilgrimstack has joined #openstack-infra07:59
*** zzzeek has quit IRC08:00
*** liusheng has quit IRC08:00
*** liusheng has joined #openstack-infra08:00
*** zzzeek has joined #openstack-infra08:01
*** bhavik1 has joined #openstack-infra08:01
*** asilenkov has joined #openstack-infra08:02
*** dteselkin has joined #openstack-infra08:03
*** igormarnat has joined #openstack-infra08:03
*** makowals_ has joined #openstack-infra08:04
*** openstackgerrit has quit IRC08:04
*** openstackgerrit has joined #openstack-infra08:05
*** matrohon has quit IRC08:05
*** makowals has quit IRC08:05
lennybwznoinsk, HI, I have some configuration issues regarding https://review.openstack.org/#/c/382262/ . After some debugging I see that my ~/.ssh/config is not configured properly.  We use Common CI Infrastructure with zuul in our CI, so we never had to configure  ~/.ssh/config. What should be configured there? http://paste.openstack.org/show/586125/08:07
wznoinskLennyb after recent changes there should be no problems to connect a metal to CI08:07
lennybwznoinsk, changes where? I am asking regarding check_jenkins_build_status.py08:08
*** matrohon has joined #openstack-infra08:10
openstackgerritMerged openstack-infra/groups: Resolve commons_activity_streams errors  https://review.openstack.org/38648408:10
*** tuanluong has quit IRC08:11
*** mfedosin has joined #openstack-infra08:11
*** nadya has joined #openstack-infra08:13
*** gildub_ has quit IRC08:15
*** isaacb has quit IRC08:16
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add a coverage job  https://review.openstack.org/38726708:17
*** makowals_ has quit IRC08:18
openstackgerritMerged openstack-infra/project-config: Adding monasca-ui to PROJECTS  https://review.openstack.org/38661308:18
*** makowals has joined #openstack-infra08:19
*** rossella_s has joined #openstack-infra08:19
openstackgerritMerged openstack-infra/project-config: Notify #openstack-dev of PTguide changes  https://review.openstack.org/38652108:20
openstackgerritMerged openstack-infra/project-config: Switch freezer python35 gate job to a voting job  https://review.openstack.org/38586308:20
*** rossella_s has quit IRC08:22
*** rossella_s has joined #openstack-infra08:23
*** ihrachys has joined #openstack-infra08:24
*** yamahata has quit IRC08:24
*** kzaitsev_mb has joined #openstack-infra08:26
*** vsaienko has quit IRC08:30
*** amitgandhinz has joined #openstack-infra08:30
*** thorst_ has joined #openstack-infra08:31
*** ihrachys has quit IRC08:33
*** thorst_ has quit IRC08:37
*** mhickey has joined #openstack-infra08:38
openstackgerritIan Wienand proposed openstack-infra/project-config: Fix gentoolkit default pkg-map  https://review.openstack.org/38786108:41
*** electrofelix has joined #openstack-infra08:42
*** john-davidge has joined #openstack-infra08:42
openstackgerritIan Wienand proposed openstack-infra/project-config: Fix unbound rc.local generation  https://review.openstack.org/38715008:42
*** lezbar has quit IRC08:43
*** ssbarnea has joined #openstack-infra08:43
*** john-davidge has quit IRC08:43
*** john-davidge has joined #openstack-infra08:43
*** lezbar__ has joined #openstack-infra08:45
*** derekh has joined #openstack-infra08:48
*** kzaitsev_mb has quit IRC08:50
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Switch ironic-multitenancy job to xenial  https://review.openstack.org/38754508:52
*** dguitarbite has joined #openstack-infra08:53
dguitarbitefungi: I want to know how to rename stable/liberty to liberty-EOL|EOL/liberty for training-labs. Just to rename the branch.08:53
*** lezbar__ has quit IRC08:55
*** lezbar has joined #openstack-infra08:56
*** markvoelker has joined #openstack-infra08:57
*** Liuqing has joined #openstack-infra08:57
*** lezbar__ has joined #openstack-infra09:00
*** lezbar has quit IRC09:00
*** degorenko|afk is now known as degorenko09:00
*** tnovacik has joined #openstack-infra09:01
*** ihrachys has joined #openstack-infra09:03
*** openstackgerrit has quit IRC09:04
*** openstackgerrit has joined #openstack-infra09:04
*** amitgandhinz has quit IRC09:05
*** winggundamth has quit IRC09:05
*** dizquierdo has joined #openstack-infra09:06
*** ssbarnea has quit IRC09:07
*** Liuqing has quit IRC09:08
*** esikachev has joined #openstack-infra09:11
*** shardy has joined #openstack-infra09:16
*** abregman|afk is now known as abregman09:16
*** bhavik1 has quit IRC09:17
openstackgerritMerged openstack-infra/devstack-gate: Remove dib-utils & diskimage-builder from PROJECTS  https://review.openstack.org/34581809:21
*** sambetts|afk is now known as sambetts09:22
*** abregman is now known as abregman|afk09:23
*** kaisers_ has quit IRC09:25
*** kaisers_ has joined #openstack-infra09:25
amrithAJaeger, do you want me to add descriptive comments into the files, or leave them in the review as comments?09:25
*** winggundamth has joined #openstack-infra09:25
amrithI would've thought adding the name 'legacy' into the descriptions would have been enough of a tell.09:26
szaherAJaeger: Hi, According to this change https://review.openstack.org/#/c/386094/ we should see 2 gates trusty and xenial why do we see only xenial ?09:28
*** e0ne_ has joined #openstack-infra09:31
AJaegeramrith: in the file itself09:34
openstackgerritamrith proposed openstack-infra/project-config: Create new trove CI jobs  https://review.openstack.org/38738609:34
amrithok, done09:34
AJaegeramrith: thanks, will review soon09:34
AJaegerszaher: I'm glad that you only see xenial ;)09:35
AJaegerszaher: trusty jobs are run only older stable branches, xenial on newton and newer. So, it's either or09:35
*** thorst_ has joined #openstack-infra09:36
AJaegeramrith: you're back to changeset 1 - you removed all the -nv work...09:37
AJaegerCompare changeset 2 and 3, it's far too large.09:37
amrithoh crap, posted commit from wrong vm :(09:37
AJaegerah ;)09:37
*** markvoelker has quit IRC09:38
* AJaeger sends some wake up coffee to amrith (or do you prefer tea?)09:38
*** markvoelker has joined #openstack-infra09:38
amrithcoffee please09:38
*** lezbar__ has quit IRC09:38
amrithAJaeger, ... http://cdn2.bigcommerce.com/n-biq04i/h3kmn/products/1528/images/1631/i_dont_do_mornings_funny_garfield_desk_sign_1391__79043.1430260350.1280.1280.jpg?c=209:39
*** lezbar has joined #openstack-infra09:39
AJaeger;)09:39
*** markvoelker has quit IRC09:40
*** jaosorior has quit IRC09:42
*** jaosorior has joined #openstack-infra09:42
lennybwznoinsk, I've found the problem in my configuration. thanks.09:42
*** thorst_ has quit IRC09:43
wznoinsklennyb: sorry, meant get back to you on that, good to hear it's sorted09:43
*** sandanar has joined #openstack-infra09:43
*** wznoinsk_ has joined #openstack-infra09:46
openstackgerritSimeon Monov proposed openstack-infra/shade: Additional domain support for grant/revoke role  https://review.openstack.org/38789209:47
*** wznoinsk has quit IRC09:47
*** ggnel_t has joined #openstack-infra09:47
*** wznoinsk_ is now known as wznoinsk09:47
szaherAJaeger: Can I force it to be on trusty and xenial as nv for newton, ocata for the moment ?09:47
*** dtantsur|afk is now known as dtantsur09:50
*** wznoinsk has quit IRC09:50
*** wznoinsk_ has joined #openstack-infra09:50
*** wznoinsk_ is now known as wznoinsk09:51
*** kzaitsev_mb has joined #openstack-infra09:52
*** claudiub|2 has quit IRC09:53
*** lezbar has quit IRC09:54
AJaegerszaher: you can, you can just add a regex for the jobs and add branch/voting for them.09:56
szaherAJaeger: Thanks09:56
*** abregman|afk is now known as abregman09:56
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Temporary disable broken job  https://review.openstack.org/38789910:00
*** amitgandhinz has joined #openstack-infra10:01
*** amotoki has quit IRC10:03
*** florianf has joined #openstack-infra10:04
*** lezbar has joined #openstack-infra10:06
openstackgerritMonty Taylor proposed openstack-infra/shade: Merge nova and neutron normalize methods  https://review.openstack.org/38790410:07
openstackgerritMonty Taylor proposed openstack-infra/shade: Document and be more explicit in normalization  https://review.openstack.org/38790510:07
*** berendt has joined #openstack-infra10:09
*** lezbar has quit IRC10:10
*** lezbar__ has joined #openstack-infra10:12
*** tuanluong has joined #openstack-infra10:12
*** isaacb has joined #openstack-infra10:19
openstackgerritChristian Berendt proposed openstack-infra/project-config: Switch kolla bandit gate job to a voting job  https://review.openstack.org/38790910:19
*** darvon has quit IRC10:20
*** pfallenop has quit IRC10:20
vponomaryovAJaeger: thanks for fast +2 of https://review.openstack.org/#/c/387899/ , looking for another +2 for merging this to unblock manila merge process10:21
openstackgerritChristian Berendt proposed openstack-infra/project-config: Switch kolla bandit gate job to a voting job  https://review.openstack.org/38790910:21
*** yanyanhu has quit IRC10:22
*** lezbar__ has quit IRC10:23
*** panda|Zz is now known as panda10:23
*** yamamoto has quit IRC10:23
*** pfallenop has joined #openstack-infra10:24
*** rossella_s has quit IRC10:28
*** rossella_s has joined #openstack-infra10:28
openstackgerritAlex Kavanagh proposed openstack-infra/project-config: Add charm-manila-generic-backend and interface  https://review.openstack.org/38791310:29
*** tkelsey has quit IRC10:30
*** ldnunes has joined #openstack-infra10:30
*** liusheng has quit IRC10:31
*** berendt has quit IRC10:31
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775010:32
openstackgerritamrith proposed openstack-infra/project-config: Create new trove CI jobs  https://review.openstack.org/38738610:33
amrithAJaeger, coffee consumed ...10:33
*** liusheng has joined #openstack-infra10:33
*** nicolasbock has joined #openstack-infra10:34
amrithdiff from #2 should be what you expect plus another suggestion I got (change legacy to LEGACY).10:34
*** amitgandhinz has quit IRC10:34
*** lezbar has joined #openstack-infra10:34
*** john-davidge has quit IRC10:35
*** berendt has joined #openstack-infra10:35
*** tuanluong has quit IRC10:35
*** john-davidge has joined #openstack-infra10:35
*** darvon has joined #openstack-infra10:35
mordredAJaeger: morning! I'm getting a weird error in a shade dsvm job - I'm wondering if an underlying config has changed that you know about10:36
mordredAJaeger: http://logs.openstack.org/05/387905/1/check/gate-shade-dsvm-functional-neutron/6e0b7f0/logs/devstacklog.txt.gz#_2016-10-18_10_23_30_64910:36
*** rhallisey has joined #openstack-infra10:37
*** jtomasek is now known as jtomasek|biab10:40
*** thorst_ has joined #openstack-infra10:43
*** lezbar has quit IRC10:43
*** chlong has quit IRC10:46
openstackgerritMarton Kiss proposed openstack-infra/groups: Security update for elysia_cron module  https://review.openstack.org/38792510:48
*** degorenko has quit IRC10:48
*** thorst_ has quit IRC10:48
*** lezbar has joined #openstack-infra10:49
*** rfolco has joined #openstack-infra10:49
*** gildub_ has joined #openstack-infra10:52
*** jkilpatr has joined #openstack-infra10:52
*** degorenko has joined #openstack-infra10:52
*** lezbar has quit IRC10:52
*** degorenko_ has joined #openstack-infra10:53
*** lezbar has joined #openstack-infra10:53
*** kzaitsev_mb has quit IRC10:55
*** kzaitsev_mb has joined #openstack-infra10:55
*** lezbar has quit IRC10:55
*** lezbar has joined #openstack-infra10:56
*** ssbarnea has joined #openstack-infra10:56
*** nadya has quit IRC10:58
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/project-config: Bumping chocolate to 100 to get more chances of tests being run on it  https://review.openstack.org/38792811:01
*** sree has quit IRC11:02
*** rtheis has joined #openstack-infra11:03
*** sree has joined #openstack-infra11:03
*** tkelsey has joined #openstack-infra11:05
*** coolsvap has quit IRC11:06
*** sree has quit IRC11:07
rcarrillocruzmordred, yolanda : you cool with ^ ? i can monitor, having a hard time to get things on chocolate due to low load and massive amount of current CI resources11:09
rcarrillocruz(morning!)11:09
openstackgerritMerged openstack-infra/system-config: Remove unused openstackci/openstackjenkins password params from controllers  https://review.openstack.org/38746111:10
*** markvoelker has joined #openstack-infra11:10
openstackgerritJordan Pittier proposed openstack-infra/shade: list_servers(): thread safety: never return bogus data.  https://review.openstack.org/38793411:10
yolanda++11:11
*** liusheng has quit IRC11:11
*** yamamoto has joined #openstack-infra11:11
*** liusheng has joined #openstack-infra11:12
*** abregman is now known as abregman|afk11:12
*** lucas-afk is now known as lucasagomes11:12
*** dprince has joined #openstack-infra11:14
*** dprince has quit IRC11:14
openstackgerritJordan Pittier proposed openstack-infra/shade: list_servers(): thread safety: never return bogus data.  https://review.openstack.org/38793411:14
*** berendt has quit IRC11:15
*** amotoki has joined #openstack-infra11:17
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775011:19
*** shardy is now known as shardy_lunch11:19
*** jkilpatr has quit IRC11:22
rcarrillocruzAJaeger: around?11:22
*** claudiub has joined #openstack-infra11:25
* amrith hides in the wings waiting for AJaeger :)11:25
rcarrillocruzheh11:28
*** kzaitsev_mb has quit IRC11:28
*** tpsilva has joined #openstack-infra11:29
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795911:30
*** amitgandhinz has joined #openstack-infra11:31
amrithrcarrillocruz, while you are waiting for AJaeger would you please give https://review.openstack.org/387386 the once over :)11:33
*** ccamacho is now known as ccamacho|lunch11:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Document and be more explicit in normalization  https://review.openstack.org/38790511:33
openstackgerritMerged openstack-infra/project-config: Bumping chocolate to 100 to get more chances of tests being run on it  https://review.openstack.org/38792811:34
rcarrillocruzdone11:35
*** jpena is now known as jpena|lunch11:38
rcarrillocruzi suppose you left legacy in caps as a reminder to later delete them11:38
AJaegeramrith: revert LEGACY change, we do not uppercase job names11:39
amrithAJaeger, will make them lower case again11:40
amrithso much for that brilliant suggestion; I'll make the change then hunt down the person who suggested it.11:40
AJaegermordred: One guess: clarkb removed some repos from PROJECTS in devstack-gate, have a look at newest job11:40
*** jkilpatr has joined #openstack-infra11:40
AJaegerrcarrillocruz: now I'm around...11:41
rcarrillocruzk, just approved the bump for chocolate11:41
rcarrillocruzi'll monitor11:41
AJaeger;)11:41
AJaegerthanks11:41
*** kzaitsev_mb has joined #openstack-infra11:42
openstackgerritamrith proposed openstack-infra/project-config: Create new trove CI jobs  https://review.openstack.org/38738611:44
amrithrcarrillocruz, AJaeger done11:44
*** thorst_ has joined #openstack-infra11:45
*** markus_z has joined #openstack-infra11:45
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726711:46
AJaegeramrith: LGTM. Hope rcarrillocruz still likes it;)11:47
rcarrillocruzheh11:47
rcarrillocruzshells look sane to me11:47
rcarrillocruz+311:47
rcarrillocruzoh, chocolate nodes being used11:47
rcarrillocruzweee11:47
amriththx AJaeger11:47
*** gouthamr has joined #openstack-infra11:48
AJaegernice11:49
*** nadya has joined #openstack-infra11:49
*** alex_xu_ has quit IRC11:50
*** dizquierdo has quit IRC11:50
*** heyongli has quit IRC11:51
*** alex_xu has joined #openstack-infra11:51
*** zul has quit IRC11:51
*** heyongli has joined #openstack-infra11:53
vsaienk0dtantsur, AJaeger: please review https://review.openstack.org/#/c/387545/ switch multitenant job to xenial11:54
*** jtomasek|biab is now known as jtomasek11:54
*** dprince has joined #openstack-infra11:55
openstackgerritMerged openstack-infra/project-config: Create new trove CI jobs  https://review.openstack.org/38738611:55
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726711:55
*** markvoelker has quit IRC11:56
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775011:57
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795911:57
*** jamesdenton has quit IRC11:57
*** flepied has quit IRC11:57
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726712:00
*** kgiusti has joined #openstack-infra12:03
*** qwertyco has joined #openstack-infra12:04
*** EricGonczer_ has joined #openstack-infra12:04
*** amitgandhinz has quit IRC12:05
rcarrillocruz2016-10-18 12:05:39.670773 | [Zuul] Job complete, result: SUCCESS12:05
rcarrillocruzweee, spotted first dsvm job run on chocolate12:06
pabelangermorning12:06
rcarrillocruzheya pabelanger12:06
*** sdake has quit IRC12:06
*** zul has joined #openstack-infra12:06
*** vsaienko has joined #openstack-infra12:06
*** asselin__ has joined #openstack-infra12:06
pabelangerhey, chocolate passed12:06
pabelangerlook at us clouding things12:07
mordredall the clouds!12:07
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726712:08
openstackgerritRob Cresswell proposed openstack-infra/project-config: Make Horizon and D_O_A Django 1.10 tests voting  https://review.openstack.org/37842612:09
*** EricGonc_ has joined #openstack-infra12:10
*** dave-mccowan has joined #openstack-infra12:10
*** EricGonczer_ has quit IRC12:10
mordredAJaeger: okie. it seems that the problem may be that we enable heat but we don't explicitly add dib-utils to our projects list12:11
*** askb has quit IRC12:12
*** markvoelker has joined #openstack-infra12:12
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726712:14
openstackgerritMonty Taylor proposed openstack-infra/project-config: Add diskimage-builder repos to shade dsvm jobs  https://review.openstack.org/38798412:15
mordredAJaeger, rcarrillocruz: ^^12:15
mordredthat should fix shade's gate12:15
rcarrillocruz+212:16
AJaegerpabelanger: woot!12:16
*** flepied has joined #openstack-infra12:16
*** edmondsw has joined #openstack-infra12:16
*** ssbarnea has quit IRC12:16
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Wait until the most recent index is available  https://review.openstack.org/38798612:17
amrithAJaeger, how long does it take before the newly updated jobs in project-config make it to the production CI? I pushed up a job in trove and ran check experimental on it; still seeing the old CI state. see https://review.openstack.org/#/c/387977/12:17
*** vsaienko has quit IRC12:18
amrithah, nice ... it updates on the fly!12:18
amrithnew jobs are running now12:18
*** trown|outtypewww is now known as trown12:19
openstackgerritMonty Taylor proposed openstack-infra/devstack-gate: Re-add dib repos to conditionally if heat is being used  https://review.openstack.org/38798812:19
mordredAJaeger, rcarrillocruz: ^^ that's another option12:19
rcarrillocruzi prefer the latter12:19
mordredAJaeger, rcarrillocruz: I blocked the first patch real quick to give us a moment to agree on which approach we prefer12:20
rcarrillocruzchances are other projects may hit that, not just shade12:20
rcarrillocruzso i'm in favor of doing that on d-g12:21
AJaegerrcarrillocruz: time-to-ready looks long for chocolate compared to vanilla, see http://grafana.openstack.org/dashboard/db/nodepool-infra-cloud12:21
*** shardy_lunch is now known as shardy12:21
openstackgerritMonty Taylor proposed openstack-infra/shade: Merge nova and neutron normalize methods  https://review.openstack.org/38790412:21
pabelangermordred: rcarrillocruz: I think that is a result of clarkb change, I know he wanted to trim down d-g12:21
mordredrcarrillocruz: that's what I was thinking12:21
rcarrillocruzyup, let's gather more data12:22
mordredpabelanger: yah. the tip of devstack-gate has the removal12:22
rcarrillocruzAJaeger: ^12:22
mordredthe shade patch there ^^ adds a depends-on to that d-g change12:22
mordredit _should_ then make things pass again12:22
mordredit's also possible that install_heat in devstack shouldn't attempt to clone those repos if it's not using them12:22
*** caowei has joined #openstack-infra12:22
pabelangerrcarrillocruz: AJaeger: interesting, I suspect it is because we are first booting nodes on the hardware and we need to download images.12:23
rcarrillocruzyeah12:23
rcarrillocruzcos12:23
rcarrillocruzthe api times12:23
rcarrillocruzare lower on chocolate12:23
rcarrillocruzi.e. the HW is better on the sl230s iirc12:23
rcarrillocruzso it may be transient till all the iamges are pulled12:23
AJaegeramrith: rule of thumb: One hour. Check status.openstack.org/zuul "Last Reconfigured" line at bottom of page12:23
*** krtaylor has joined #openstack-infra12:23
AJaegerrcarrillocruz, mordred: Please discuss with clarkb !12:24
rcarrillocruzpabelanger: btw, i've set the quota on chocolate to 100 nodes12:24
pabelangerrcarrillocruz: ack12:24
pabelangerwas that with cloud-launcher?12:24
rcarrillocruzwe'll have time to tune later on12:24
amrithAJaeger, thanks. The jobs were picked up and are failing :( Have to figure out why ... right now having issues getting to the nodes (known ipv6 issues from home)12:24
rcarrillocruzno, cos my change is not passing the gate :/12:24
pabelangerok12:24
AJaegeramrith: just wait, log files will be available soon ;)12:24
rcarrillocruzi'm hitting the issue on centos nodes on osic1 not resolving12:24
rcarrillocruzis the change that ianw has in gate to fix12:24
AJaegerrcarrillocruz: this one? https://review.openstack.org/38715012:25
pabelangerwhat issue is that?12:25
rcarrillocruzevery time i'm unlucky to get a centos on osic1 in the gate, it fails to pull from pypi12:25
mordredAJaeger, rcarrillocruz, pabelanger: for completeness: https://review.openstack.org/38799012:25
rcarrillocruzAJaeger: yep, that one12:25
rcarrillocruzso if we land https://review.openstack.org/#/c/387150/3 , that'd be great12:27
rcarrillocruzcos the launcher has been stuck 3-4 days due to osic issues :/12:27
rcarrillocruzerm, not osic, but the centos thing12:27
mordredrcarrillocruz, pabelanger, AJaeger, clarkb: https://review.openstack.org/#/q/topic:fix-heat-dsvm ... there are three ways to fix this. the more I look at it, the more I like the devstack patch12:28
*** asselin has joined #openstack-infra12:28
*** asselin__ has quit IRC12:28
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python3[45] and coverage jobs  https://review.openstack.org/38726712:28
rcarrillocruzi would rule out the shade only patch12:30
rcarrillocruzand i can understand the rationale to cleanup d-g, in the end we are even trying to pull stuff out on roles too... the less clutter the better12:30
pabelangerrcarrillocruz: looking12:30
rcarrillocruzi suppose clarkb will like the devstack patch too12:30
rcarrillocruzthx12:30
*** raildo has joined #openstack-infra12:30
pabelangerrcarrillocruz: -1, because we are using a deprecated flag12:32
rcarrillocruzoh?12:32
rcarrillocruzlet me see12:32
pabelanger-a12:32
openstackgerritamrith proposed openstack-infra/project-config: fix path for build-image job  https://review.openstack.org/38799912:32
pabelangershould be -e12:32
rcarrillocruzk, let's fix inline12:32
rcarrillocruzi'll push an edit12:33
pabelangerbut logic looks to be correct12:33
*** claudiub|2 has joined #openstack-infra12:33
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/project-config: Fix unbound rc.local generation  https://review.openstack.org/38715012:33
rcarrillocruzedited12:34
amrithrcarrillocruz, AJaeger, https://review.openstack.org/387999 will fix it. will wait for all the jobs to finish, review logs and then ping you; sorry for the bother12:34
rcarrillocruz+212:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775012:34
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - TEST  https://review.openstack.org/38795912:34
pabelangerrcarrillocruz: I'm not sure this is your issue however, based on comments, this looks to only be a problem for Fedora12:34
*** asselin_ has joined #openstack-infra12:35
pabelangerunless, the is the chmod 0755 that is stopping systemd from running it12:35
*** vsaienko has joined #openstack-infra12:35
pabelangerokay, +312:35
pabelangerwe can give it a try12:35
rcarrillocruzianw put on commit message "This should fix the odd issues we see for centos & fedora on OSIC's12:35
rcarrillocruzipv6 only nodes"12:35
pabelangerok12:35
rcarrillocruz++12:36
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add python35 and coverage jobs  https://review.openstack.org/38726712:36
*** claudiub has quit IRC12:36
*** EricGonc_ has quit IRC12:36
pabelangermordred: want to try and land your zuul-launcher patches this morning?12:37
*** sree has joined #openstack-infra12:38
*** jpena|lunch is now known as jpena12:38
*** asselin has quit IRC12:38
*** esikachev has quit IRC12:40
mordredpabelanger: ooh. that's a great idea12:40
*** krtaylor has quit IRC12:41
*** gildub_ has quit IRC12:41
mordredpabelanger: what do you think - just land the whole stack then do the restarts?12:42
*** sree has quit IRC12:42
mordredpabelanger: maybe I should queue up a revert patch just in case it all goes horribly wrong12:42
*** amoralej is now known as amoralej|lunch12:43
pabelangermordred: Ya, a revert might be good. But either a single launcher or all, I have the time to help debug issues12:43
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bagpipe: add tempest, cover, python3 and periodic jobs  https://review.openstack.org/38544512:45
mordredpabelanger: ohh. see - there you are being smart suggesting restarting a single launcher12:45
*** martinkopec has joined #openstack-infra12:46
*** burgerk has joined #openstack-infra12:46
*** Naeil has quit IRC12:46
rcarrillocruzotl12:46
rcarrillocruzbbiab12:47
openstackgerritMerged openstack-infra/project-config: Fix unbound rc.local generation  https://review.openstack.org/38715012:48
*** esikachev has joined #openstack-infra12:50
*** tiswanso has joined #openstack-infra12:50
*** tiswanso has quit IRC12:50
openstackgerritMonty Taylor proposed openstack-infra/zuul: Revert command module rework  https://review.openstack.org/38800412:51
openstackgerritMerged openstack-infra/groups: Security update for elysia_cron module  https://review.openstack.org/38792512:51
*** tiswanso has joined #openstack-infra12:51
*** markvoelker has quit IRC12:53
openstackgerritKhai Do proposed openstack-infra/system-config: Add a nodepool to zuul-dev.o.o  https://review.openstack.org/38757912:54
*** piet_ has quit IRC12:55
*** Hal has joined #openstack-infra12:55
*** Qiming_ has joined #openstack-infra12:56
*** Qiming_ has quit IRC12:56
mordredpabelanger: ok- I clicked +A on the series, and also -2'd the revert patch12:56
pabelangermordred: ack12:57
*** vsaienko has quit IRC12:57
*** strigazi1AFK is now known as strigazi12:58
*** ccamacho|lunch is now known as ccamacho12:58
vsaienk0ianw, pabelanger: could you please help to merge chain of 3 patches started from https://review.openstack.org/#/c/364830/ they are needed for ironic multinode enablement already has +2 from, thanks!12:59
*** tmorin has joined #openstack-infra13:01
*** amitgandhinz has joined #openstack-infra13:02
tmorinhi AJaeger: sorry for the confusion and missed attempts about the new net-.. jobs, and thanks for your comments, I think the jobs should be ok now13:02
*** hrubi has quit IRC13:02
*** asselin has joined #openstack-infra13:03
openstackgerritMonty Taylor proposed openstack-infra/shade: Additional domain support for grant/revoke role  https://review.openstack.org/38789213:03
*** jaosorior is now known as jaosorior_mtg13:04
openstackgerritMerged openstack-infra/project-config: fix path for build-image job  https://review.openstack.org/38799913:04
*** xyang1 has joined #openstack-infra13:04
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Wait until the most recent index is available  https://review.openstack.org/38798613:05
*** krtaylor has joined #openstack-infra13:05
*** vsaienko has joined #openstack-infra13:05
*** mriedem has quit IRC13:05
*** asselin_ has quit IRC13:06
*** jcoufal has joined #openstack-infra13:06
*** bauzas has quit IRC13:09
*** bauzas has joined #openstack-infra13:10
*** zz_dimtruck is now known as dimtruck13:10
*** mdrabe has joined #openstack-infra13:10
*** Marx314 has joined #openstack-infra13:11
*** drifterza has quit IRC13:12
pabelangermordred: testrepository.subunit was > 50 MB of uncompressed data!!! says the gate13:13
pabelangerguess we are logging lots of data now13:13
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add infracloud-chocolate tree to Cacti  https://review.openstack.org/38801113:14
*** hurgleburgler has joined #openstack-infra13:14
*** Naeil has joined #openstack-infra13:14
*** Goneri has joined #openstack-infra13:17
*** amitgandhinz has quit IRC13:17
*** Hal has quit IRC13:18
*** amitgandhinz has joined #openstack-infra13:18
mordredpabelanger: for which job?13:18
*** vsaienko has quit IRC13:18
*** markvoelker has joined #openstack-infra13:19
*** vsaienko has joined #openstack-infra13:20
*** nadya has quit IRC13:20
*** psachin` has quit IRC13:20
pabelangermordred: sorry, that was the start of the stack for async changes in zuul13:21
pabelangerhttp://logs.openstack.org/03/369603/7/gate/gate-zuul-python27-db-ubuntu-xenial/b7710d0/console.html13:21
mordredrcarrillocruz: the devstack-gate patch definitely fixes the shade gate: https://review.openstack.org/#/c/387904/13:21
rcarrillocruz++13:21
mordredpabelanger: nice13:21
mordredpabelanger, rcarrillocruz: I propose we land it for now, and then if the devstack change lands we can revert it13:22
rcarrillocruzwfm13:22
rcarrillocruzyou have my +2 on it already13:23
mordredrcarrillocruz: thanks!13:23
*** jamesdenton has joined #openstack-infra13:23
mordredpabelanger: hrm. I'm not sure what's changed about that13:23
mordredalso - why do we have mysql and postgres in zuul's bindep.txt ?13:24
*** cardeois has joined #openstack-infra13:24
pabelangerya, first time I see that error13:24
pabelangerneed to install them for db tests13:24
pabelangerIIRC13:24
mordredpabelanger: zuul doesn't have a db though13:25
mordredpabelanger: is an empty bindep.txt file valid?13:25
pabelangerHmm, that is right.13:25
pabelangerlet me check history13:25
*** isaacb has quit IRC13:25
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Wait until the most recent index is available  https://review.openstack.org/38798613:26
*** jcoufal has quit IRC13:26
openstackgerritMonty Taylor proposed openstack-infra/zuul: Stop installing mysql and postgres  https://review.openstack.org/38801913:26
mordredpabelanger: ^^ I figured I'll just try with a patch :)13:26
*** baoli_ has joined #openstack-infra13:27
pabelangermordred: Ah, we were using gate-zuul-python27-db13:27
pabelangernot any more13:27
pabelangerhttps://review.openstack.org/#/c/325519/13:27
mordredgotit13:27
mordredpabelanger: so it's just leftover from the transition13:28
pabelangerya13:28
mordredwoot13:28
*** ssbarnea has joined #openstack-infra13:28
rcarrillocruzpabelanger: telnetting a job on centos osic1, it pulled from pypi just fine13:28
rcarrillocruz\o/13:29
openstackgerritMonty Taylor proposed openstack-infra/zuul: Stop installing mysql and postgres  https://review.openstack.org/38802013:29
mordredforward ported to v3 branch ^^13:29
openstackgerritMonty Taylor proposed openstack-infra/shade: Document and be more explicit in normalization  https://review.openstack.org/38790513:29
odyssey4meFYI, just registered https://github.com/eliben/pycparser/issues/15113:31
odyssey4meit looks like a new wheel was uploaded for pycparser and it's broken, again13:31
AJaegermordred: empty bindep.txt file is fine13:31
*** eharney has joined #openstack-infra13:31
*** abregman|afk is now known as abregman13:32
mordredAJaeger: woot!13:32
AJaegermordred: but you need a chang efor project-config to use gate-zuul-python27 instead of gate-zuul-python27-db13:33
openstackgerritKhai Do proposed openstack-infra/system-config: Add a nodepool to zuul-dev.o.o  https://review.openstack.org/38757913:33
AJaegermordred: shall I sent one?13:33
*** sree has joined #openstack-infra13:33
*** jtomasek is now known as jtomasek|afk13:33
*** vikrant has quit IRC13:35
*** ramishra has joined #openstack-infra13:35
*** matt-borland has joined #openstack-infra13:35
openstackgerritMonty Taylor proposed openstack-infra/project-config: Stop running database variants of zuul tests  https://review.openstack.org/38802213:35
mordredAJaeger: like that ^^ ?13:35
*** sdake has joined #openstack-infra13:36
*** aviau has quit IRC13:36
AJaegermordred: exactly - +213:36
*** jcoufal has joined #openstack-infra13:36
*** aviau has joined #openstack-infra13:37
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit  https://review.openstack.org/37511313:37
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Add StoryBoard integration for graph commands  https://review.openstack.org/38511213:37
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Add Jira integration for graph commands  https://review.openstack.org/38521713:37
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Refactor launchpad code into bug_tracker  https://review.openstack.org/38519913:37
*** vsaienko has quit IRC13:38
dguitarbiteAJaeger: Any suggestions on how to rename the older branch to make it EOL?13:39
*** hrubi has joined #openstack-infra13:39
asselinZara, when you get a chance, please review "Add StoryBoard integration for graph commands  https://review.openstack.org/385112".13:39
*** zhurong has joined #openstack-infra13:40
AJaegerdguitarbite: we don't rename. We add a tag and delete the branch13:40
*** annegentle has joined #openstack-infra13:40
*** ssbarnea has quit IRC13:40
AJaegerdguitarbite: any infra-root can do that for you, just ask to tag stable/X branch as EOL.13:40
AJaegerOr better ask to retire stable/X13:40
*** esikachev has quit IRC13:40
*** ssbarnea has joined #openstack-infra13:41
mordredAJaeger: you know - I don't see any eol branches for nova past juno-eol - did we stop doing that there?13:41
Zaraasselin: thank you, will take a look in a bit!13:41
fungiinfra-root (and everyone else): i'm back home and on the network again. today will be mostly trying to catch up on the past 11 days and preparing for a day full of meetings, so don't expect me to be super responsive outside the meetings themselves13:41
AJaegermordred: http://git.openstack.org/cgit/openstack/nova/tag/?h=kilo-eol13:42
rcarrillocruz\o/13:42
mordredfungi: welcome back!!! land all my patches and pay attention to me now!!!!13:42
rcarrillocruzhappy to see you back and safe fungi13:42
AJaegerfungi: welcome back! How was your vacation?13:42
mordredAJaeger: fascinating. that does not exist in my local repo13:42
AJaegermordred: not a branch, a tag!13:42
mordredAJaeger: yah - I don't have any tags in my local nova repo past juno-eol13:43
mordredAJaeger: and doing git fetch is not fixing it13:43
mordredI mean, it's not important ot me13:43
mordredbut it's still weird13:43
*** amoralej|lunch is now known as amoralej13:44
AJaegermordred: did you disable fetching of tags? Try git fetch --tags13:45
*** amitgandhinz has quit IRC13:45
*** jaosorior_mtg is now known as jaosorior13:46
*** ssbarnea has quit IRC13:46
*** amitgandhinz has joined #openstack-infra13:46
*** tlian has joined #openstack-infra13:46
*** dizquierdo has joined #openstack-infra13:47
*** ramishra has quit IRC13:47
*** ggnel_t has quit IRC13:47
pabelangerrcarrillocruz: likely unrelated, we need to rebuild images to get ianw patch13:47
*** mtanino has joined #openstack-infra13:48
AJaegerpabelanger: do you want to include prometheanfire Gentoo change for that build? To see that everything works...13:48
pabelangerAJaeger: sure, should be fine now13:48
yolandahi, i'm seeing ci crashing again with pycparser bug13:48
AJaegerhttps://review.openstack.org/#/c/387114/ is the change13:48
yolanda2016-10-18 13:47:19.429768 |       File "/home/jenkins/workspace/gate-tripleo-quickstart-linters-ubuntu-xenial/.tox/linters/lib/python2.7/sre_compile.py", line 583, in compile13:48
yolanda2016-10-18 13:47:19.429797 |         "sorry, but this version only supports 100 named groups"13:48
yolanda2016-10-18 13:47:19.429828 |     AssertionError: sorry, but this version only supports 100 named groups13:48
yolanda2016-10-18 13:47:19.429841 |13:48
rcarrillocruzah ofc, it was a dib element fix13:49
yolandai saw the pin was reverted some time ago, anyone knows why is it failing again?13:49
*** ssbarnea has joined #openstack-infra13:49
rcarrillocruzthx13:49
*** kzaitsev_mb has quit IRC13:49
*** ativelkov has joined #openstack-infra13:49
*** kzaitsev_mb has joined #openstack-infra13:49
AJaegerpabelanger: +2A for Gentoo nodepool13:49
fungiAJaeger: vacation was great. i hadn't taken any time off yet this year, so it was great to be able to unplug completely for a little while13:50
*** kzaitsev1mb has joined #openstack-infra13:50
*** kzaitsev1mb has quit IRC13:50
*** hamzy has quit IRC13:50
*** EricGonczer_ has joined #openstack-infra13:50
AJaegerThat was well deserved then ;)13:50
AJaegerGlad that you could outrun the storm ;)13:50
kzaitsev_mbwe're having wierd devstak failures in the gate — smth related with heat (gate job says clonning not allowed when installing heat) is it a known problem or are we hitting something new?13:51
AJaegerkzaitsev_mb: known problem, mordred is on it but we're waiting for clarkb to figure out best option AFAIK13:52
kzaitsev_mbAJaeger: thanks for the info =)13:52
yolandaah AJaeger , glad to see that this is being covered13:52
sdakehey folks - it looks like newton gate is busted: http://logs.openstack.org/26/388026/1/check/gate-kolla-dsvm-build-centos-binary-centos-7-nv/4d9eecb/console.html#_2016-10-18_13_47_10_45391113:52
AJaegerkzaitsev_mb: we removed a few projects from default PROJECTS13:52
sdakeany tips?13:53
sdakeor is that what you already talking about13:53
AJaegersdake: see yolanda 's comment from 5 mins ago in backscroll13:53
openstackgerritMerged openstack-infra/project-config: Stop running database variants of zuul tests  https://review.openstack.org/38802213:53
yolandaAJaeger, that's not same issue as mordred is looking?13:53
AJaegeryolanda: no, that's a completely different one13:53
yolandathe one i see is the old pycparser one, that was solved by pinning to 2.13, but i saw the change was reverted some time ago. I don't know why it fails again13:53
AJaegeryolanda: not seen yours before13:54
AJaegerwelcome back, fungi. We just broke everything ;)13:54
*** ralonsoh_ has joined #openstack-infra13:54
AJaegerwe just don't know how we did it :)13:54
*** burgerk has quit IRC13:54
sdakefungi just in time ;-)13:54
yolandathe one i see looks related with https://review.openstack.org/#/c/381518/13:55
sdakeya - master and newton gates are completely bust13:55
*** ralonsoh has quit IRC13:55
sdakealhtough mitaka seems ok13:55
AJaegeryolanda: sorry, can't help with that13:55
*** Hal has joined #openstack-infra13:55
yolandayep, looks as a requirement problem. But i don't know why pin was reverted, and what was the proper fix13:55
yolandadims, do you have more feedback about it?13:56
*** ssbarnea has quit IRC13:56
odyssey4mesdake that issue has been registered upstream: https://github.com/eliben/pycparser/issues/15113:56
*** ssbarnea has joined #openstack-infra13:56
AJaegeryolanda: pin was reverted since a broken wheel was uploaded and used by our CI.13:56
AJaegerThe source code was fine, just the wheel was broken.13:56
odyssey4methe wheel has been pulled already, but the infra mirror will need to be re-synced to pull it off13:56
yolandamm, and now it happens again13:56
dimsupstream removed their wheel and then fungi removed all instances from our mirrors13:57
dimsouch13:57
AJaegerDid they push again a broken wheel? Or done another release?13:57
yolandathe error is the same, but not sure about release13:57
yolandapypi says 2.15 again, but uploaded on 2016-10-18...so today13:57
rcarrillocruz:-)13:58
pabelangerhttps://pypi.python.org/pypi/pycparser13:58
pabelangeryup13:58
rcarrillocruzlulz13:58
pabelanger2.15 today13:58
odyssey4meAJaeger yolanda see the bug https://github.com/eliben/pycparser/issues/15113:58
AJaegerproject-config changes break because of this as well13:58
odyssey4methere is discussion in there about how to work around it and the cause, etc13:58
*** tiswanso has quit IRC13:58
yolandayes, same bug came back13:58
* AJaeger will be back later13:58
funginot sure what symptoms you're seeing in this case... if it's similar to the one from before i went out of town, bandersnatch (if working properly) updates our pypi mirrors on deletion with no trouble. however, the current design of our wheel mirror is purely additive and copies wheels from pypi when available. so you end up needing to delete the copied wheel(s) out of afs manually13:59
AJaegeryolanda: sent a #status notice to inform people about hte problem13:59
yolandai need to step out for a meeting just now...13:59
yolandabe back in half an hour i guess13:59
AJaegerpabelanger: could you do a #status, please?14:00
* AJaeger has to step out as well14:00
*** Guest62846 is now known as med_14:00
fungithere was an e-mail thread on the -dev ml where i gave a (brief) post-mortem of the issue from that incident14:00
*** med_ has quit IRC14:00
*** med_ has joined #openstack-infra14:00
*** mriedem has joined #openstack-infra14:00
pabelangerjust left comment on PR about tagging 2.15.1 and not deleting an existing wheel14:00
*** EricGonczer_ has quit IRC14:00
*** ramishra has joined #openstack-infra14:01
fungiyeah, deletion is generally of questionable benefit. though i still think our wheel mirror builder needs a bit of a redesign to avoid copying wheels from pypi14:01
pabelanger++14:02
fungiin a better design, we would only populate our wheel mirrors with wheels we've built14:02
fungisince pip will find the ones on our pypi mirror if they already exist, so we only need ones that don't exist in pypi on our separate wheel mirror14:02
*** mhickey has quit IRC14:02
pabelangerAJaeger: status notice?14:02
pabelanger#status notice We are away of pycparser failures in the gate and working to address the issue.14:03
openstackstatuspabelanger: sending notice14:03
-openstackstatus- NOTICE: We are away of pycparser failures in the gate and working to address the issue.14:05
pabelangerwoah14:05
pabelangerhttp://mirror.dfw.rax.openstack.org/pypi/simple/1-844-291-6706-dell-printer-technical-support-phone-number/14:05
pabelangerthat is new14:05
pabelangeror is that a package name14:05
mordredwow14:05
*** gordc has joined #openstack-infra14:05
pabelangerthere is a bunch14:05
sdakeodyssey4me thanks - just woke up here (pst timezone)14:05
mordredhttps://pypi.python.org/pypi/1-844-291-6706_DELL_Printer_Technical_Support_Phone_Number/84414:06
pabelangerya14:06
mordreddstufft: nice spam you got there14:06
dimsmordred : LOL14:06
*** kgiusti1 has joined #openstack-infra14:06
mordredlike, that's impressive14:06
pabelangerindeed14:06
rcarrillocruzpabelanger: yeah, i read it too fast, it pulled ok from mirror, failed afterwards to pull ansible . What I'm puzzled about this problem is why it can pull with pip just fine outside of a venv and we see this problem when creating the tox venvs14:06
openstackstatuspabelanger: finished sending notice14:06
*** mptacekx has joined #openstack-infra14:06
rcarrillocruzanyway, i'll wait for the dib image rebuilds14:07
mordredrcarrillocruz: the earlier pips are maybe just to our mirror and not to pypi, so they're not leaving the cloud?14:07
*** kgiusti has quit IRC14:07
mordredrcarrillocruz: and the ansible clone is from github which is off the cloud?14:07
* mordred guessing14:07
rcarrillocruzright...14:08
rcarrillocruzmordred: lol14:08
*** tiswanso has joined #openstack-infra14:08
rcarrillocruzhttp://mirror.regionone.infracloud-vanilla.openstack.org/pypi/simple/14:08
rcarrillocruzlook at the beginning of the page14:08
rcarrillocruzmoar spam pls!14:08
mptacekxHi, we just encountered on intel-nfv-ci issue with pycparser 2.15, devstack is crashing with error "AssertionErrror: sorry, but this version only supports 100 named groups"14:08
mptacekxanyone else affected ?14:08
pabelangeryes, we are working on fixing the issue14:08
mptacekxsame issue appeared few weeks back with pycparser 2.14 and it had to be rebuild14:09
pabelangerhttps://github.com/eliben/pycparser/issues/15114:09
*** bauzas is now known as bauzas_off14:09
mptacekxthanks ! cool14:09
mordredrcarrillocruz: I like the packages "1" and "0-0"14:09
mordredoh - that's actually 0-._.-._.-._.-._.-._.-._.-014:10
pabelangerI guess they moved on from attaching wiki.o.o14:10
openstackgerritStefan Majewsky proposed openstack-infra/infra-manual: explain how to install git-review on Arch Linux  https://review.openstack.org/38803614:10
rcarrillocruzheh14:10
rcarrillocruzso mordred , that's the issue, but why it goes out the tox pip install ? we have ansible in our mirrors too...14:10
rcarrillocruz?14:10
mordredrcarrillocruz: we have jobs that install ansible master14:11
mordredif it's a thing that installs it from released version, then yeah, it should hit our mirror14:11
*** xarses has quit IRC14:11
*** asselin_ has joined #openstack-infra14:11
* rcarrillocruz goes check the version from requirements.txt to compare what is in th emirror14:12
*** lucasagomes is now known as lucas-hungry14:12
*** asselin has quit IRC14:13
mordredrcarrillocruz: wehich job is failing?14:13
*** florianf has quit IRC14:14
szaherAJaeger: I am seeing strange failures https://review.openstack.org/#/c/388021/1 and https://review.openstack.org/#/c/386990/ after merging https://review.openstack.org/#/c/386094/14:14
rcarrillocruzhttps://review.openstack.org/#/c/386556/14:14
rcarrillocruzi'm confused, the launcher has 2.1.2.0 pin for ansible14:14
rcarrillocruzand we have that on the mirrors14:14
AJaegerszaher: what strange failures? The one that the #status notice just got broadcast?14:15
*** jaosorior has quit IRC14:16
mordredrcarrillocruz: oh - but the place where the earlier pip stuff succeeds is in devstack before neutron breaks networking14:16
mordredrcarrillocruz: so that's consistent with the way those fail14:16
*** asselin_ has quit IRC14:16
rcarrillocruzahk14:16
*** sdake has quit IRC14:16
*** sdake_ has joined #openstack-infra14:17
*** mhickey has joined #openstack-infra14:17
*** nadya has joined #openstack-infra14:17
*** qwertyco has quit IRC14:17
*** burgerk has joined #openstack-infra14:18
*** rbrndt has joined #openstack-infra14:22
*** abregman is now known as abregman|afk14:22
tmorinhi everyone14:23
*** hamzy has joined #openstack-infra14:23
tmorinwould someone know how I can pass an option to the coverage tool through a "python setup.py test --coverage" call in tox.ini ?14:23
*** ssbarnea has quit IRC14:24
*** ssbarnea has joined #openstack-infra14:25
jlvillalpabelanger: I wonder if the PyPi folks know someone spammed their repository?14:25
*** piet_ has joined #openstack-infra14:26
jlvillaltmorin: Not sure but you might try.  python setup.py testr --coverage --  --YOUR_ARGS_HERE14:27
*** tiswanso has quit IRC14:27
*** eharney has quit IRC14:27
*** rossella_s has quit IRC14:28
*** ramishra has quit IRC14:28
*** ramishra_ has joined #openstack-infra14:28
szaherAJaeger: I can't get what's wrong in here http://logs.openstack.org/21/388021/1/check/gate-osbackup-freezer-dsvm-ubuntu-xenial/4274820/console.html ?14:28
pabelangerjlvillal: we have a ping out to dstufft14:28
*** rossella_s has joined #openstack-infra14:29
jlvillalpabelanger: Thanks14:29
*** asselin has joined #openstack-infra14:29
*** amotoki has quit IRC14:30
dmelladohey guys14:32
dmelladois there any patch up there yet for the pycparser issue?14:32
*** asselin_ has joined #openstack-infra14:32
*** vsaienko has joined #openstack-infra14:32
*** abregman|afk is now known as abregman14:33
tmorinjlvillal: in fact, I've just found a solution by looking at pbr source code, it's already calling the coverage option I want (--source)14:33
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add chocolate hosts to Cacti  https://review.openstack.org/38804514:33
jlvillaltmorin: Cool :)14:33
tmorinjlvillal: the trick was that the pbr option is --coverage-package-name while I wanted to specify multiple package names14:33
*** markvoelker has quit IRC14:34
tmorinbut since its passed as-is into coverage --source option, I now know that I can give it multiple package names14:34
*** flepied1 has joined #openstack-infra14:34
*** asselin has quit IRC14:35
mordredpabelanger: it's feeling like the pycparsing change ate in to our nice quiet morning - maybe let's push restarting zuul worker until tomorrow morning?14:35
openstackgerritRicardo Carrillo Cruz proposed openstack-infra/system-config: Add chocolate hosts to Cacti  https://review.openstack.org/38804514:35
pabelangermordred: sure14:35
pabelangerfor some reason cffi is pulling in 2.15, even thought we are capped at 2.14 in upper-constraints14:35
pabelangerdigging more into it14:36
mordredpabelanger: cool. that way we can debug liesurely14:36
*** flepied has quit IRC14:36
mordredpabelanger: oh lovely14:36
mordredpabelanger: is cffi pulling it in via setup_requires?14:36
pabelangerchecking14:36
pabelanger1.8.3 was released yesterday it seems14:36
pabelangermordred: I see pycparser in install_requires14:37
mordredweird14:38
mordredI would expect constraints to work right14:38
fungihave a tox log showing pip's decision process there?14:38
pabelangerlet me find one14:38
*** mfedosin has quit IRC14:39
fungiusually that's how i suss it out, after some unfortunate amount of time wasted staring blankly at it and pulling at my hair a bit14:39
*** sdake_ has quit IRC14:39
pabelangerhttp://logs.openstack.org/92/384892/8/check/gate-tripleo-quickstart-docs-ubuntu-xenial/add8cd0/tox/venv-1.log14:39
mordredthat invocation does not mention constraints14:40
pabelangerAh, right14:40
pabelanger1 src14:41
*** sree has quit IRC14:41
pabelangerhttp://logs.openstack.org/02/384202/5/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c653441/logs/devstacklog.txt.gz14:41
pabelangerdid14:41
pabelangerbut need to find tox log14:41
*** eharney has joined #openstack-infra14:41
*** tkelsey has quit IRC14:41
mordredpabelanger: I disagree14:41
mordredpabelanger: that also doesn't use constraints14:42
mordredhttp://logs.openstack.org/02/384202/5/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c653441/logs/devstacklog.txt.gz#_2016-10-18_14_23_46_58314:42
*** nherciu has quit IRC14:42
pabelangermordred: what I was looking at http://logs.openstack.org/02/384202/5/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c653441/logs/devstacklog.txt.gz#_2016-10-18_14_21_30_50314:42
rcarrillocruzreviews for adding chocolate to cacti pls : https://review.openstack.org/#/c/388011/ and https://review.openstack.org/#/c/388045/14:43
*** mhickey has quit IRC14:43
mordredpabelanger: yes! I agree that that does mentoin constraints and also shows the issue14:44
*** rajinir has joined #openstack-infra14:45
*** sdake has joined #openstack-infra14:46
jlvillalpabelanger: FYI: Regarding PyPi package spam. I did create an issue: https://sourceforge.net/p/pypi/support-requests/680/14:46
pabelangerjlvillal: great, thanks14:46
* mordred does a doubletake at the url14:46
*** kgiusti1 has quit IRC14:46
*** kgiusti has joined #openstack-infra14:47
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: ubuntu-xenial set to non-voting  https://review.openstack.org/38805514:47
ihrachyspabelanger: should we block 1.8.3 then?14:47
fungimordred: pabelanger: skimming that log, looks like it's probably an unversioned setup_requires of cryptography14:47
*** claudiub|2 has quit IRC14:48
mordredihrachys: yah. it's starting to look that way14:48
*** vsaienko has quit IRC14:48
fungier, of cffi via cryptography, sorry14:48
szaherAJaeger: https://review.openstack.org/#/c/388055/ Can you please review this one ?14:48
*** ssbarnea has quit IRC14:49
pabelangerfungi: right, seen that too14:49
*** psachin` has joined #openstack-infra14:49
*** ssbarnea has joined #openstack-infra14:50
*** yamahata has joined #openstack-infra14:50
rcarrillocruzbah, pycparser -1 me14:51
rcarrillocruzi'll go take my son now i guess14:51
*** xgerman has quit IRC14:51
openstackgerritThomas Morin proposed openstack-dev/pbr: coverage package name option, doc improvement  https://review.openstack.org/38805914:52
ihrachyspabelanger: mordred: https://review.openstack.org/388060 ?14:52
fungihrm, looking at the cffi source, it's just an install_requires14:52
*** apetrich has quit IRC14:53
*** psachin` has quit IRC14:54
ihrachysfungi: yeah. actually, it's same in 1.8.214:55
pabelangerihrachys: was going to say, I think 1.8.2 will have the same issue14:55
*** kun_huang has quit IRC14:56
*** mhickey has joined #openstack-infra14:56
*** Hal has quit IRC14:56
*** puiterwijk|cld has quit IRC14:56
ihrachyspabelanger: so then blocking the new cffi should not help14:56
fungicryptography 1.5.2 has a setup_requires of cffi>=1.4.1 though https://github.com/pyca/cryptography/blob/1.5.2/setup.py#L5814:56
*** mriedem has quit IRC14:56
fungii think that's the problem (pycparser is a transitive install_requires of a setup_requires)14:56
clarkbmordred: pabelanger that was ianw's change. But yes we should be trying to push that into the jobs or I guess in this case add them in if some known dep uses them14:56
pabelangerfungi: ah, I think you are right14:57
fungiso upper-constraints is applied to cryptography, but not to cffi because it's a setup_requires, and so the constraints option doesn't make it down the chain to pycparser when (unconstrained) cffi drags it in14:57
mordredclarkb: I think the devstack change is the most correct fix14:58
*** mriedem has joined #openstack-infra14:58
*** amcguign has joined #openstack-infra14:58
mordredclarkb: https://review.openstack.org/#/c/387990/14:58
clarkbmordred: oh even better14:59
mordredd-g patch is just a bandaid until that lands14:59
*** kun_huang has joined #openstack-infra14:59
mordredfungi: I agree with your assessment14:59
*** rackertom has quit IRC14:59
*** hamzy has quit IRC15:00
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: ubuntu-xenial set to non-voting  https://review.openstack.org/38805515:00
*** puiterwijk|cld has joined #openstack-infra15:00
*** erlon has quit IRC15:02
*** hamzy has joined #openstack-infra15:02
*** amcguign has quit IRC15:03
*** dirk has quit IRC15:04
yolandaback from meeting15:04
*** kun_huang has quit IRC15:04
clarkbmordred: reviewed your devstack change15:04
clarkbmordred: needs a small change15:05
ihrachysfungi: couldn't we expect install_requires to apply constraints?15:06
ihrachystrying to understand if it's a bug in pip15:06
*** tnovacik has quit IRC15:06
fungiihrachys: it's applied as a command-line option15:06
mordredclarkb: thanks! fixed15:07
fungiso the "outer" `pip install -c upper-constraints.txt cryptography` triggers an `easy_install cffi` which in turn triggers a new `pip install pycparser` or something like that15:07
fungibut easy_install doesn't know that the15:07
*** mhickey has quit IRC15:08
fungi"outer" pip install which called it had command line options that need to get applied to the "inner" `pip install` it triggers15:08
ihrachyswe could replace pip with a wrapper that would always pass the constraints file as an argument ;)15:08
fungior maybe there's an envvar we could use instead of the command-line switch?15:08
*** lukasa has joined #openstack-infra15:08
fungithough also i don't think the "inner" pip install is really pip. it's probably also easy_install15:09
*** rackertom has joined #openstack-infra15:09
fungithe log at least doesn't show the phrasing i'd expect in pip's output15:09
*** petems has quit IRC15:09
ihrachysyou mean the one installing pycparser? yeah, that would sadly be the most logical course of action15:09
fungiinstead of "Collecting pycparser" it says "Searching for pycparser" (and then grabs an sdist rather than a wheel)15:10
fungiso smells like pip install cryptography is triggering easy_install cffi (because of setup_requires) which then does an easy_install pycparser (because easy_install doesn't know to use pip for install_requires?)15:11
*** lucas-hungry is now known as lucasagomes15:12
*** dirk has joined #openstack-infra15:12
dhellmannfungi : that sounds likely15:12
*** annegentle has quit IRC15:12
*** petems has joined #openstack-infra15:13
*** tkelsey has joined #openstack-infra15:13
lukasafungi: Yeah, pip doesn't manage setup_requires15:13
ihrachysfungi: we could also have easy_install as a wrapper calling pip ;)15:13
dhellmannis there actually a bug in pycparser?15:13
*** xgerman has joined #openstack-infra15:14
fungidhellmann: looks like it http://logs.openstack.org/02/384202/5/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c653441/logs/devstacklog.txt.gz#_2016-10-18_14_21_39_51715:14
*** kun_huang has joined #openstack-infra15:14
lukasadhellmann: Yeah15:14
lukasaBut cryptography is triggering it15:14
fungi"AssertionError: sorry, but this version only supports 100 named groups"15:14
lukasaI'm chatting to the cryptography devs15:14
*** EricGonczer_ has joined #openstack-infra15:14
lukasaBut the problem seems to be that their bindings are causing pycparser to build a really, really big regex.15:14
dhellmannok15:14
lukasaFor the moment the cryptography folks think this is a pycparser bug, but they're going to try to debug it just to validate that assertion.15:15
lukasa(And because right now no-one else is debugging it)15:15
*** EricGonczer_ has quit IRC15:15
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251515:15
*** abregman has quit IRC15:15
*** EricGonczer_ has joined #openstack-infra15:15
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251515:16
clarkbmordred: I like that the heat in projects list issue isn't actualyl what is breaking everything15:16
*** adrian_otto has joined #openstack-infra15:16
*** nicolasbock has quit IRC15:16
mordredclarkb: ikr?15:17
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: ubuntu-xenial set to non-voting  https://review.openstack.org/38805515:17
mordredfungi: we can work around by pre-installing cffi so that easy_install doens't do it15:18
mordredfungi: for some versions of "we can"15:18
fungiit's worth a shot at least15:18
fungipresumably just a straight-up `pip install cryptography` exhibits the problem at this point?15:19
lukasamordred: +1 that should work15:19
fungii will admit i really haven't had time to try and recreate the issue myself. still catching up15:19
clarkbfungi: http://logs.openstack.org/90/387990/2/check/gate-tempest-dsvm-multinode-full-ubuntu-xenial/c74b4e5/logs/devstacklog.txt.gz#_2016-10-18_15_14_54_564 yes looks like it15:21
dhellmannhmm, I was able to pip install cryptography on my local system15:21
dhellmannit looks like it is using some cached wheels15:21
*** nicolasbock has joined #openstack-infra15:21
openstackgerritJames E. Blair proposed openstack-infra/zuul: Default test log level to DEBUG except for testr  https://review.openstack.org/38807315:21
dhellmannhttp://paste.openstack.org/show/586224/15:21
pilgrimstackhi all15:22
pilgrimstackWe see that infra is not using OVH intensivley15:22
jeblairclarkb, mordred, pabelanger: can you look at https://review.openstack.org/388073 and its parent (which i just approved)15:22
pilgrimstackThere is a special reason?15:22
pilgrimstackSome issue or it's just you have less needs15:23
*** edtubill has joined #openstack-infra15:23
clarkbpilgrimstack: we use our clouds in a ratio of cloudquota:totalquota. Right now total quota is fairly large so every cloud is using less total resources as a whole. That said I think we may still be debugging some image upload issues to bhs1 and are just using gra1 for now15:24
clarkbpabelanger: ^ is that correct?15:24
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: Fixed tempest regex  https://review.openstack.org/38807615:24
*** cshastri has quit IRC15:24
pabelangerclarkb: pilgrimstack: Ya, we have BHS1 disabled right now, but think we have a fix for our image upload issues. https://review.openstack.org/#/c/386254/15:25
mordredjeblair: ++15:25
openstackgerritMerged openstack-infra/zuul: Lower the log level in tests  https://review.openstack.org/26336415:26
pilgrimstackok :)15:26
*** baoli_ has quit IRC15:26
pabelangerjeblair: ++15:26
*** nherciu has joined #openstack-infra15:26
*** claudiub has joined #openstack-infra15:27
clarkbjeblair: reviewed15:27
*** tnovacik has joined #openstack-infra15:27
openstackgerritMichael Johnson proposed openstack-infra/project-config: Adds a functional test job for Octavia  https://review.openstack.org/38807815:27
*** tiswanso has joined #openstack-infra15:28
*** mriedem1 has joined #openstack-infra15:28
*** cardeois has quit IRC15:29
jeblairclarkb: nice catch, thanks15:29
pabelangermordred: clarkb: I'd be game if we wanted to cherry-pick 386254 to nodepool.o.o and re-eanble BHS1 to test.  Otherwise, if we want to schedule a shade release, I can start the process to bring ovh-bhs1 back online15:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Default test log level to DEBUG except for testr  https://review.openstack.org/38807315:29
*** mriedem has quit IRC15:29
jeblairclarkb, mordred, pabelanger: ^15:29
mordredpabelanger: if we can get the gate happy, I'd lke to make a shade release today or tomorrow - but I'm also ok with doing a cherry-pick and testing in case we need to do a _different_ fix15:30
*** armax has joined #openstack-infra15:31
clarkbjeblair: that looks better I have approved15:31
*** Rockyg has joined #openstack-infra15:31
pabelangermordred: okay, let me bring ovh-bhs1 back online15:31
anteayapilgrimstack: also keep in mind that this week and next are some of our slowest weeks in the cycle15:31
anteayapilgrimstack: overall cloud usage should pick up after summit15:31
sandanaron some of the gate tests on openstack/driverlog project i see this error "this version only supports 100 named groups" on cryptography package install -- http://logs.openstack.org/69/388069/2/check/gate-driverlog-pep8-ubuntu-xenial/45f518b/console.html .. any clue on this error ?15:32
anteayaclarkb: the logstash job queue graph looks angry15:32
pabelangerclarkb: mordred: Oh, I did the right thing when I removed it from nodepool: https://review.openstack.org/#/c/37603815:32
pabelangersandanar: yes, we are debugging it now15:33
sandanar@pabelanger .. thanks for the information15:33
pabelangersandanar: https://github.com/eliben/pycparser/issues/151 is the upstream issue15:33
clarkbanteaya: any changes to mqtt again?15:33
*** vhosakot has joined #openstack-infra15:33
*** nadya has quit IRC15:33
anteayaI do not know, I can look into it15:33
clarkbanteaya: oh actually it could be related to pycparser making jobs fail really fast15:34
clarkbthat will generate a lot of events in a short period if time. ES clsuter says it is green, I think we should just wait a few to see if it goes away once pycparser is addressed15:34
openstackgerritMarton Kiss proposed openstack-infra/groups: Upgrade Drupal Core to 7.51  https://review.openstack.org/38808115:34
sdakehate to be a pest, is there an eta on the gates?15:34
*** ilyashakhat_mobi has joined #openstack-infra15:34
*** krtaylor has quit IRC15:34
anteayaclarkb: ah that sounds more likely15:34
pabelangernot yet15:34
sdaketrailing projects tag today15:34
*** dizquierdo has quit IRC15:34
mordredsdake: yah - we know the issue, but have not yet found a solution that'll stem the bleeding15:35
anteayaclarkb: the latest mosquito version was released in august15:35
*** cardeois has joined #openstack-infra15:35
sdakemordred roger15:35
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: Fixed tempest regex  https://review.openstack.org/38807615:35
*** vsaienko has joined #openstack-infra15:38
*** mdrabe has quit IRC15:39
*** nadya has joined #openstack-infra15:39
*** nadya has quit IRC15:39
odyssey4mepabelanger do you have any idea how we get the upper constraints file added to ansible functional tests? http://logs.openstack.org/72/388072/3/check/gate-openstack-ansible-pip_install-ansible-func-ubuntu-xenial/eec95a5/console.html#_2016-10-18_15_36_55_72590115:39
openstackgerritMerged openstack-infra/zuul: Default test log level to DEBUG except for testr  https://review.openstack.org/38807315:39
pabelangerodyssey4me: Hmm, I'd have to look. I think AJaeger has do a lot of work on it15:40
*** martinkopec has quit IRC15:40
*** mhickey has joined #openstack-infra15:41
pabelangerodyssey4me: I think you need to be setup for requirements job15:41
dhellmannfungi : I lost track of the conversation, is there a plan for this pycparser thing? Do we think we can do anything about it ourselves, or do we need to wait for more upstream analysis?15:41
odyssey4mepabelanger we already are for our repositories, so I don't think that's it15:41
fungidhellmann: mordred hypothesized that we could try to find some way to preinstall cffi so that pycparser gets installed via constraints in pip rather than transitively via easy_install, and then easy_install will in theory find it's already present?15:42
lukasadhellmann: Yeah, fungi and mordred's plan is the one that openstack can most easily do15:43
dhellmannfungi, mordred : ok, I verified that that approach does work, but I don't relish the idea of having to edit everyone's tox.ini files to implement it15:43
lukasaThat's definitely the downside. =(15:43
dhellmannit's a shame that our pypi mirror doesn't honor our constraints list15:43
pabelangerodyssey4me: you run the proposal bot against your repos?15:43
odyssey4mepabelanger yep15:43
pabelangerk15:43
fungidhellmann: what do you mean about our pypi mirror honoring the constraints list?15:43
dhellmannfungi : well, if the mirror only included the versions of things that we like based on our constraint list, we could globally ignore this even with the pip/setuptools hand-off issue, right?15:44
*** chandankumar has quit IRC15:44
pabelangerodyssey4me: I would expect that to add the upper-constraints.txt to your repo15:44
odyssey4mefungi mordred dhellmann yeah, having to manually add a requirement to install first everywhere is a rather intenable solution15:44
pabelangerodyssey4me: but I might be wrong15:44
fungidhellmann: ahh, so a special subset mirror that only includes versions found in a particular constraints list (one for each branch of requirements i suppose)?15:45
odyssey4mepabelanger I don't think so - the file is present in all the linter/docs jobs... so something must add it15:45
dhellmannfungi : or one mirror that merged all of the constraints or something (obviously I'm thinking out loud)15:45
fungiodyssey4me: well, we've done it before, but odds are in the weeks it will take to get merged everywhere the problem will be solved upstream15:45
odyssey4mefungi yeah, and we need to tag a final newton RC today :/15:45
*** vhosakot has quit IRC15:46
pabelangerodyssey4me: looks like zuul-git-prep-upper-constraints might do it15:46
*** vhosakot has joined #openstack-infra15:46
dhellmannfungi : so we might mirror 4 versions of cffi, and depend on the local constraints file to choose the "right" one for a given job15:46
dhellmannbut we would only ever add new things to the mirror when a constraints list is updated15:47
*** mdrabe has joined #openstack-infra15:47
fungiwhich makes it also harder to test constraints changes (would have to not use that mirror)15:47
dhellmannI'm not sure how practical that would be to implement, though15:47
dhellmannyeah, that, too15:47
clarkbdhellmann: we have avoided doing that because then our mirrors become required15:47
dhellmannclarkb: good point15:48
*** ilyashakhat_mobi has quit IRC15:48
fungithat's also true. anyone who expects, e.g., tox to work locally needs to use our special mirror or make one of their own rather than just relying to public pypi15:48
pabelangerodyssey4me: Ya, switching ansible-role-jobs to zuul-git-prep-upper-constraints should be what is needed15:48
*** adrian_otto has quit IRC15:48
odyssey4mepabelanger aha :) shall I push a patch up for that, or will you?15:48
dhellmannfungi : yeah. it would also be a lot of work to route around what is really an issue in the install tool15:48
pabelangerodyssey4me: So, I'm not sure what will happen on ansible roles that don't use requirements today15:49
odyssey4mepabelanger well, all that does is put the file there - currently the env var is set, but there's no file15:49
pabelangerodyssey4me: I mean, we can try15:49
fungipabelanger: odyssey4me: constraints lists are allowed to be incomplete, so in theory it would still do what it's supposed to15:49
odyssey4meto actually use upper constraints requires other work15:49
pabelangerokay, cool15:50
pabelangerso, lets try it15:50
lukasav2.16 of pycparser just got pushed15:50
*** mugsie has quit IRC15:50
fungiif there is a constraint for what you're asking pip to install, it will apply it. if you ask pip to install something for which there is no constraint listed, it will just follow its normal version selection rules for that particular package15:50
fungilukasa: thanks for the heads up!15:50
*** xarses has joined #openstack-infra15:50
lukasaThe author believes that may have fixed the issue15:51
lukasaWorth testing15:51
lukasafungi: np, happy to help15:51
*** dimtruck is now known as zz_dimtruck15:51
mriedem12.16 works for me15:52
dhellmann2.16 works for me, too15:52
*** mriedem1 is now known as mriedem15:52
*** andreas_s has quit IRC15:52
fungiin theory it should show up on our mirrors momentarily15:52
*** zhurong has quit IRC15:52
odyssey4mepabelanger your patch or mine? :)15:52
pabelangerodyssey4me: go for it, so I can +215:52
odyssey4meokie dokey - up in a few mins15:53
dhellmannfungi : I have a few release jobs that need to be retriggered. Will you have some time to help with that today? maybe after lunch?15:53
openstackgerritMichael Johnson proposed openstack-infra/project-config: Adds a functional test job for Octavia  https://review.openstack.org/38807815:53
dguitarbiteAJaeger: Allright, I will do so very soon. Thanks :).15:53
fungidhellmann: how many? i'm still trying to catch up and prep for today's meetings... just got home last night15:53
dhellmannfungi : https://etherpad.openstack.org/p/retagging-tripleo-rc315:53
fungidhellmann: so... ~5? i should be able to squeeze that in15:55
*** e0ne_ has quit IRC15:55
*** e0ne has joined #openstack-infra15:55
dhellmannfungi : I wasn't sure if the follow-up jobs also had to be manually triggered since they were skipped15:55
dhellmannbut yeah, 5 deliverables15:56
*** erlon has joined #openstack-infra15:56
pabelangerrcarrillocruz: AJaeger: infracloud-chocolate time to ready is starting to flatten out15:56
*** caowei has quit IRC15:56
*** annegentle has joined #openstack-infra15:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add upper constraints prep to ansible-role-jobs  https://review.openstack.org/38808715:58
odyssey4mepabelanger ^ not sure if that's the right place for it, and if that's complete15:58
fungidhellmann: hopefully it's safe to just reenqueue these since there were no successful uploads15:58
pabelangerodyssey4me: you can remove the original zuul-git-prep15:58
dhellmannfungi : sounds good15:58
*** panda is now known as panda|bbl15:58
*** lukasa has left #openstack-infra15:59
*** adrian_otto has joined #openstack-infra15:59
fungidhellmann: though for a couple of those you only have a releasenotes or release announcement failure linked... did those fail to build/upload tarballs too?15:59
dhellmannfungi : those are all of the failures I got by email. I'll double check the tarball state16:00
fungithe three with tarball job failures, i can just reenqueue the ref for those existing tags16:00
*** esikachev has joined #openstack-infra16:00
fungiso will be simple enough16:00
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add upper constraints prep to ansible-role-jobs  https://review.openstack.org/38808716:02
*** mptacekx has quit IRC16:03
*** sdake_ has joined #openstack-infra16:03
mordredI go to get a sandwich and I come back and pycparser is fixed!16:05
*** sambetts has quit IRC16:05
fungiclearly sandwiches are the answer16:05
mordredyah. I should get sandwiches every time we have an issue16:05
pleia2mmm sandwiches16:06
pabelangersudo mordred find sandwiches FTW16:06
*** sdake has quit IRC16:06
*** irtermit- is now known as irtermit16:06
*** irtermit is now known as irtermite16:06
yolandaah, so they pushed a new pycparser version now16:08
pabelangeryolanda: I just tested with your original patch16:08
pabelangerlooks to be fixed16:08
RockygBribed with the idea of sandwiches16:08
yolandacool16:08
pabelanger#status notice pycparser 2.16 released to fix assertion error from today.16:09
openstackstatuspabelanger: sending notice16:09
*** markus_z has quit IRC16:10
dhellmannfungi : I have confirmed that of the 6 items tagged as part of that release, only 3 of them failed to upload tarballs.16:10
-openstackstatus- NOTICE: pycparser 2.16 released to fix assertion error from today.16:10
openstackgerritMerged openstack-infra/groups: Upgrade Drupal Core to 7.51  https://review.openstack.org/38808116:11
openstackstatuspabelanger: finished sending notice16:12
*** matrohon has quit IRC16:13
*** clenimar has joined #openstack-infra16:13
fungidhellmann: oh! i see why i was confused. you have jobs for a couple of those deliverables listed in multiple sections16:13
dhellmannfungi : do I?16:13
fungiso there are actually only failed jobs for three deliverables, and all of those failed their tarball jobs16:13
openstackgerritMerged openstack-infra/zuul: Tidy up README.rst  https://review.openstack.org/38563416:13
openstackgerritMerged openstack-infra/zuul: Add a note to README.rst about contributing to Zuul v3  https://review.openstack.org/38563516:13
dhellmannhmm, I copied those lists from the emails I got16:14
dhellmannoh, I see, tripleoclient has 2 blocks16:14
fungipuppet-tripleo and python-tripleoclient have two sections each16:14
*** sambetts_ has joined #openstack-infra16:15
*** sandanar has quit IRC16:15
mriedemhas anyone put up a blacklist patch to g-r for pycparser 2.15 yet?16:15
*** sambetts_ is now known as sambetts16:16
*** ssbarnea has quit IRC16:17
dhellmannfungi : does that look better?16:17
*** tmorin has quit IRC16:17
*** ssbarnea has joined #openstack-infra16:17
*** yaume has quit IRC16:18
SwansonAnyone seeing cliff fail to install or is it a network thing on my end?16:18
pabelangermriedem: you are the first to mention it16:18
mriedemok i'll do it16:18
*** markvoelker has joined #openstack-infra16:19
fungidhellmann: yep, so i'll retrigger the puppet-tripleo-announce-release job, and then reenqueue the refs for the other three that can be completely rerun16:19
mriedemnevermind,16:20
mriedempycparser isn't in g-r16:20
mriedemshould it be?16:20
dhellmannfungi : thanks!16:20
mordredmriedem: it's a transitive dep16:20
mriedemright, so it's fine to be in g-r right?16:21
mordredI don't think it's a problem to put it in there16:21
dhellmannit would be fine to add it, but it would not have helped in this case because of the way it's being installed16:21
mordredyah16:21
mriedemok, i'll just leave it out16:22
*** jtomasek|afk is now known as jtomasek16:22
*** ssbarnea has quit IRC16:22
*** krtaylor has joined #openstack-infra16:22
*** ssbarnea has joined #openstack-infra16:23
*** inc0 has joined #openstack-infra16:23
*** hamzy has quit IRC16:23
openstackgerritMerged openstack/diskimage-builder: Add zypper-minimal element  https://review.openstack.org/38157516:24
*** ssbarnea has quit IRC16:24
openstackgerritMerged openstack-infra/project-config: Revert "Remove ovh-bhs1 from nodepool.yaml"  https://review.openstack.org/37603816:24
*** Guest85855 has quit IRC16:25
pabelangermordred: when you have time, would appreciate some feedback on https://review.openstack.org/#/c/385608/ ansible-playbook support for diskimage-builder.  I left some comments in line16:25
*** ssbarnea has joined #openstack-infra16:26
openstackgerritMerged openstack/diskimage-builder: Add opensuse-minimal element  https://review.openstack.org/38157616:27
openstackgerritMerged openstack-infra/project-config: Add Gentoo support to nodepool  https://review.openstack.org/38711416:27
*** ssbarnea has quit IRC16:28
*** ssbarnea has joined #openstack-infra16:29
*** kzaitsev_mb has quit IRC16:29
*** mfedosin has joined #openstack-infra16:29
greghaynespabelanger: re: your comment there about not wanting to have to make an element - What we can do is have support for running ansible.d inside of an element then make an element that is something like simple-playbook which, if you add it to your elements, lets you point it at a playbook via env var16:31
*** kzaitsev_mb has joined #openstack-infra16:31
greghaynesi.e. ANSIBLE_PLAYBOOK=myplaybook.yml disk-image-create distro simple-playbook16:31
*** ssbarnea has quit IRC16:32
*** ssbarnea has joined #openstack-infra16:32
*** e0ne_ has joined #openstack-infra16:32
*** e0ne has quit IRC16:32
andreykurilin_hi folks! Is it possible to change an email for openstackid.org ?16:33
*** baoli has joined #openstack-infra16:34
*** maeker has joined #openstack-infra16:34
*** yamahata has quit IRC16:34
*** knangia has joined #openstack-infra16:35
*** hamzy has joined #openstack-infra16:36
fungidhellmann: just to confirm, the instack-undercloud tag was supposed to be a release candidate not a full release, right?16:38
openstackgerritLin Yang proposed openstack-infra/project-config: Rename rsc to valence  https://review.openstack.org/38133216:38
pabelangergreghaynes: Ya, I think that would be a good middle ground.  In fact, I'll update the patch to support that too16:39
pabelangerand add a test16:39
*** degorenko has quit IRC16:39
clarkbanteaya: it actually looks like we may have hit a processing wall yesterday which caused things to pile up. At least on worker02 things appear to have crashed on crm114 timing out16:39
*** degorenko_ is now known as degorenko16:40
clarkbanteaya: trying to go through the workers to see if that is consistent and restart services where they have died16:40
clarkbso far 2/2 on crm114 timing out at ~2030UTC yesterday16:41
*** tqtran has joined #openstack-infra16:42
*** jpich has quit IRC16:44
cloudnullafternoons16:44
clarkbthey seem to crash relatively quickly after restarting too on the same issue. Will likely need more digging16:44
clarkbcloudnull: good morning/afternoon. Semi random question. Is osic-cloud1 booting instances off of ceph like cloud8 does? I am guessing no based on the ssd and spinning disk flavors?16:45
*** psilvad has joined #openstack-infra16:45
cloudnullclarkb: no we're using local disk16:45
cloudnullhowever we're going to add a ceph AG in the after-now16:45
*** abregman has joined #openstack-infra16:47
*** mtanino has quit IRC16:47
cloudnullthe data that we're gathering on the specific flavors has been great and I think adding yet another AG + Flavor will provide some great metrics.16:48
clarkblooks like crm114 times out on the very long wheel build job console logs16:48
fungiclarkb: so does my browser. maybe that's a sign we should make those jobs less verbose16:49
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: [WIP]: Add janitor thread  https://review.openstack.org/38758316:49
*** eharney has quit IRC16:50
*** ssbarnea has quit IRC16:50
*** rcernin has quit IRC16:51
*** claudiub has quit IRC16:51
*** Apoorva has joined #openstack-infra16:51
clarkbfungi: its about 150k lines of data, not sure about byte count just yet but am checking16:52
*** Apoorva has quit IRC16:52
clarkbfungi: 32MB uncompressed16:52
fungiyeah, that doesn't seem like a useful log length to me16:53
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: [WIP]: Add janitor thread  https://review.openstack.org/38758316:55
*** srobert has joined #openstack-infra16:57
mgagnewhat's the best way to scale zuul-merger? can more worker be spawn or do I need a whole new VM ?16:57
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: [WIP]: Add janitor thread  https://review.openstack.org/38758316:58
*** amitgandhinz has quit IRC16:59
*** Apoorva has joined #openstack-infra16:59
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: [WIP]: Add janitor thread  https://review.openstack.org/38758316:59
*** ssbarnea has joined #openstack-infra16:59
*** degorenko is now known as degorenko|afk16:59
openstackgerritKhai Do proposed openstack-infra/system-config: Add a nodepool to zuul-dev.o.o  https://review.openstack.org/38757916:59
Shrews*sigh*16:59
clarkbmgagne: I think it shoudl be fine to use multiple on a single host as long as they modify and serve discrete git repos16:59
clarkbmgagne: we use separate VMs to simplify doing ^ but should be possible on a single host I think16:59
*** jkilpatr has quit IRC16:59
mgagneclarkb: this means multiple /var/lib/zuul/git folders?17:00
mgagneright17:00
*** ralonsoh_ has quit IRC17:01
*** eharney has joined #openstack-infra17:01
*** vsaienko has quit IRC17:01
*** dtantsur is now known as dtantsur|afk17:01
*** derekh has quit IRC17:01
*** ssbarnea has quit IRC17:02
clarkbmgagne: and possibly multiple apache vhosts or serving at a different root path for each17:02
mgagne+117:02
*** Swami has joined #openstack-infra17:02
clarkbmgagne: so https://merger01.foo.com/ or https://merger.foo.com/01/ etc17:02
*** chandankumar has joined #openstack-infra17:02
dstufftUm, I saw a highlight of my name here but it scrolled past so I can't see what folks said17:03
dstufftwhat's up17:03
gordcpabelanger: just curious: does this patch work against all jobs that enable heat or does it require a specific conf option? https://review.openstack.org/#/c/387988/17:04
*** ssbarnea has joined #openstack-infra17:04
fungidstufft: i think it was mostly just a heads up of package spam someone spotted on pypi. i think an issue got opened in the tracker for it17:04
dstufftAh okay17:04
dstufftI'll take a look17:04
gordcasking because that dib-utils patch broke telemetry jobs17:04
gordc(i believe)17:05
dstufftthanks!17:05
*** jpena is now known as jpena|off17:06
fungidstufft: sure thing. looks like it was mostly the scheme where phishers get their scam phone numbers to show up in search engines for support on popular software17:06
openstackgerritClark Boylan proposed openstack-infra/puppet-log_processor: Handle errors in previously working filters  https://review.openstack.org/38812217:06
dstufftfun17:06
clarkbanteaya: fungi and ^ is the workaround on the log processor side17:06
dstufftI'll have to figure out how tod elete them all, hopefully they're the same user17:06
dstufftI'm a bit swamped right this second :/17:07
fungifigured17:07
clarkbgordc: it should work against any jobs that have heat enabled. THen https://review.openstack.org/#/c/387990/ is the long term fix17:07
*** chandankumar has quit IRC17:07
fungidstufft: according to my channel log, jlvillal opened https://sourceforge.net/p/pypi/support-requests/680/ about it17:08
*** e0ne_ has quit IRC17:09
jlvillaldstufft: Sadly, I don't think they are all the same user :(17:09
gordcclarkb: awesome! thanks for the info. i'll track those patches then17:09
*** ssbarnea has quit IRC17:10
fungijlvillal: dstufft: we get similar spam on our mediawiki instance, and the usual pattern is that they create a new account through some openid provider, use it to post stuff they want search engines to index, and never reuse the same account. it's easy enough to script new accounts for every spam unfortunately17:10
*** edtubill has quit IRC17:11
fungiand easier for them than figuring out if the account has been discovered and disabled/deleted17:11
anteayaclarkb: thank you, reviewed17:12
*** chandankumar has joined #openstack-infra17:12
*** ihrachys has quit IRC17:13
*** chandankumar has quit IRC17:14
*** chandankumar has joined #openstack-infra17:14
*** gouthamr has quit IRC17:14
*** jkilpatr has joined #openstack-infra17:14
*** vsaienko has joined #openstack-infra17:14
*** trown is now known as trown|lunch17:15
AJaegerodyssey4me: change zuul-git-prep to zuul-git-prep-upper-constraints in ansible-role-jobs.yaml17:15
openstackgerritMerged openstack-infra/zuul: Fix confusing debug message in configloader.py  https://review.openstack.org/38696017:15
jlvillalTrying to get patches merged so we can get Ironic multi-node testing working. This devstack-gate one is next in line:  https://review.openstack.org/#/c/364830   Currently has one +2. Help appreciated :)17:18
clarkbjlvillal: I have reviewed the whole stack right?17:18
jlvillalclarkb: You have and you are awesome :)17:18
*** BobBall is now known as BobBall_AWOL17:18
clarkbok, just double checking I didn't miss anything /me goes back to unsticking logstash17:18
*** yamahata has joined #openstack-infra17:18
jlvillalThanks17:19
*** gouthamr has joined #openstack-infra17:19
odyssey4meAJaeger thanks :) I see you found the review17:19
*** cody-somerville has joined #openstack-infra17:19
*** cody-somerville has joined #openstack-infra17:19
mordreddstufft: they do mostly start with 1-800, 1-844, 1800 or 1844 at least17:20
anteayathey are so considerate17:21
*** mhickey has quit IRC17:21
*** Naeil has quit IRC17:21
clarkbjlvillal: oh, because you are here. sc68cal is working on a way to put the service lists for multinode into the features matrix17:21
clarkbjlvillal: just something to be aware of as hopefully changing soon and will need porting for your things17:21
*** annegentle has quit IRC17:21
jlvillalclarkb: Sweet. I had wondered why that wasn't done before. But didn't look too hard at it.17:21
*** csomerville has quit IRC17:22
jlvillalsc68cal: Hopefully you can incorporate our Ironic service list into the matrix17:22
jlvillalsc68cal: As an FYI: https://review.openstack.org/#/c/36861117:22
*** sambetts is now known as sambetts|afk17:23
*** maeker has quit IRC17:24
*** zz_ja is now known as zz_zz_ja17:25
AJaegerodyssey4me: 388072 is the wrong way for doing constraints ;(17:26
AJaegerodyssey4me: let me show how to do it properly...17:26
AJaegerodyssey4me: ah, now read the comment completely...17:27
*** maeker has joined #openstack-infra17:28
openstackgerritMerged openstack-infra/project-config: Add upper constraints prep to ansible-role-jobs  https://review.openstack.org/38808717:28
odyssey4meAJaeger yeah, I'll fix up our tox config once the jobs have the file17:28
odyssey4meheh, speaking on which :)17:28
odyssey4me*of17:28
AJaegerodyssey4me: Just fixed it - https://review.openstack.org/388146 - but will need recheck once the change is in effect17:29
openstackgerritElizabeth K. Joseph proposed openstack-infra/project-config: Add entercloud regions to Nodepool  https://review.openstack.org/38814917:31
pleia2clarkb: ^17:31
odyssey4meAJaeger yeah, thanks :)17:32
*** yamahata has quit IRC17:32
*** jbernard has quit IRC17:33
*** beisner has quit IRC17:33
*** jeblair has quit IRC17:33
*** SotK has quit IRC17:33
*** Kiall_ has quit IRC17:33
*** lucasagomes has quit IRC17:33
*** njohnston has quit IRC17:33
*** Guest61315 has quit IRC17:33
*** auggy has quit IRC17:33
*** pkoniszewski has quit IRC17:33
*** nt has quit IRC17:33
*** lindsayk has quit IRC17:33
*** jistr has quit IRC17:33
*** lbragstad has quit IRC17:33
*** bstinson has quit IRC17:33
*** irtermite has quit IRC17:33
*** karhtikp_ has quit IRC17:33
*** dolphm has quit IRC17:33
*** nikhil has quit IRC17:33
*** khappone has quit IRC17:33
*** raddaoui has quit IRC17:33
*** sslypushenko has quit IRC17:33
*** kgalanov has quit IRC17:33
*** johnsom has quit IRC17:33
*** csatari has quit IRC17:33
*** jpmaxman has quit IRC17:33
*** jaranovich has quit IRC17:33
*** aglarendil has quit IRC17:33
*** cargonza has quit IRC17:33
*** jamespage has quit IRC17:33
*** asingh_ has quit IRC17:33
*** zaro has quit IRC17:33
*** trash has quit IRC17:33
*** wcriswell has quit IRC17:33
*** flaper87 has quit IRC17:33
*** Anticimex has quit IRC17:33
*** aimeeu has quit IRC17:33
*** skath has quit IRC17:33
*** igorbelikov has quit IRC17:33
*** ewindisch has quit IRC17:33
*** DuncanT has quit IRC17:33
*** coreyob has quit IRC17:33
*** ggherdov- has quit IRC17:33
*** geekinutah has quit IRC17:33
*** vern has quit IRC17:33
*** Guest66666 has quit IRC17:33
*** jgriffith has quit IRC17:33
*** redrobot has quit IRC17:33
*** sfinucan has quit IRC17:33
*** dstufft has quit IRC17:33
*** scottda has quit IRC17:33
*** EmilienM has quit IRC17:33
*** timburke has quit IRC17:33
*** SpamapS has quit IRC17:33
*** Jokke_ has quit IRC17:33
*** antonym has quit IRC17:33
*** Madasi has quit IRC17:33
*** jklare has quit IRC17:33
*** jasondotstar has quit IRC17:33
*** kencjohnston has quit IRC17:33
*** notmyname has quit IRC17:33
*** michaelxin has quit IRC17:33
*** sulo has quit IRC17:33
*** kevinbenton has quit IRC17:33
*** weshay has quit IRC17:33
*** gtmanfred has quit IRC17:33
*** sbalukoff has quit IRC17:33
*** quade has quit IRC17:33
*** mhayden has quit IRC17:33
*** boris-42 has quit IRC17:33
*** patrickeast has quit IRC17:33
*** iremizov has quit IRC17:33
*** mgkwill has quit IRC17:33
*** betherly has quit IRC17:33
*** ashtokolov has quit IRC17:33
*** fbo has quit IRC17:33
*** madorn has quit IRC17:33
*** greghaynes has quit IRC17:33
*** Apsu has quit IRC17:33
*** dims has quit IRC17:33
*** automagically has quit IRC17:33
*** sigmavirus has quit IRC17:33
*** mancdaz has quit IRC17:33
*** Ng has quit IRC17:33
*** edwarnicke has quit IRC17:33
*** mattfarina has quit IRC17:33
*** warpc has quit IRC17:33
*** tchaypo has quit IRC17:33
*** TheJulia has quit IRC17:33
*** asaprykin has quit IRC17:33
*** jpena|off has quit IRC17:33
*** verdurin has quit IRC17:33
*** jroll has quit IRC17:33
*** briancline has quit IRC17:33
*** johnthetubaguy has quit IRC17:33
*** cburgess has quit IRC17:33
*** tonyb has quit IRC17:33
*** cloudnull has quit IRC17:33
*** briancli1e has joined #openstack-infra17:33
*** jasondotstar has joined #openstack-infra17:33
*** dolphm_ has joined #openstack-infra17:33
*** beisner has joined #openstack-infra17:33
*** jgriffith_ has joined #openstack-infra17:33
*** Guest66666 has joined #openstack-infra17:33
*** Anticimex has joined #openstack-infra17:33
*** irtermite has joined #openstack-infra17:33
*** jbernard has joined #openstack-infra17:33
*** automagically has joined #openstack-infra17:33
*** tonyb has joined #openstack-infra17:33
*** trash has joined #openstack-infra17:33
*** dims has joined #openstack-infra17:33
*** Madasi has joined #openstack-infra17:33
*** Kiall has joined #openstack-infra17:33
*** Vivek has joined #openstack-infra17:33
*** dstufft has joined #openstack-infra17:33
*** cburgess has joined #openstack-infra17:33
*** kencjohnston has joined #openstack-infra17:33
*** Jokke_ has joined #openstack-infra17:33
*** jeblair has joined #openstack-infra17:33
*** Apsu has joined #openstack-infra17:33
*** Jokke_ has quit IRC17:33
*** Jokke_ has joined #openstack-infra17:33
*** jamespag` has joined #openstack-infra17:33
*** antonym has joined #openstack-infra17:33
*** mancdaz_ has joined #openstack-infra17:33
*** pkoniszewski has joined #openstack-infra17:33
*** greghaynes has joined #openstack-infra17:33
*** redrobot has joined #openstack-infra17:33
*** jklare_ has joined #openstack-infra17:33
*** SpamapS has joined #openstack-infra17:33
*** mancdaz_ is now known as mancdaz17:33
*** SotK has joined #openstack-infra17:33
*** johnthetubaguy_ has joined #openstack-infra17:33
*** antonym has quit IRC17:33
*** antonym has joined #openstack-infra17:33
*** quade has joined #openstack-infra17:34
*** timburke has joined #openstack-infra17:34
*** verdurin has joined #openstack-infra17:34
*** weshay has joined #openstack-infra17:34
*** verdurin has quit IRC17:34
*** verdurin has joined #openstack-infra17:34
*** cloudnull has joined #openstack-infra17:34
*** gtmanfred has joined #openstack-infra17:34
*** dolphm_ is now known as dolphm17:34
*** coreyob has joined #openstack-infra17:34
*** jamespag` has quit IRC17:34
*** jamespag` has joined #openstack-infra17:34
*** redrobot is now known as Guest4136617:34
*** Vivek is now known as Guest9184117:34
*** gtmanfred has quit IRC17:34
*** gtmanfred has joined #openstack-infra17:34
*** karthikp has joined #openstack-infra17:34
*** sulo has joined #openstack-infra17:34
*** johnthetubaguy_ is now known as johnthetubaguy17:34
*** aricg has left #openstack-infra17:34
*** vern has joined #openstack-infra17:34
*** lucasagomes has joined #openstack-infra17:34
*** EmilienM has joined #openstack-infra17:34
*** bstinson has joined #openstack-infra17:34
*** jistr has joined #openstack-infra17:34
*** notmyname has joined #openstack-infra17:34
*** mhayden has joined #openstack-infra17:34
*** jroll has joined #openstack-infra17:34
*** madorn has joined #openstack-infra17:34
*** jpena|off has joined #openstack-infra17:34
*** _sigmavirus24 has joined #openstack-infra17:34
*** khappone has joined #openstack-infra17:34
*** haaj has joined #openstack-infra17:35
*** sbalukoff has joined #openstack-infra17:35
*** EmilienM has quit IRC17:35
*** EmilienM has joined #openstack-infra17:35
*** amitgandhinz has joined #openstack-infra17:35
*** njohnston has joined #openstack-infra17:36
*** sfinucan has joined #openstack-infra17:36
*** fkautz has quit IRC17:36
*** dham1 has quit IRC17:36
*** kevinbenton has joined #openstack-infra17:36
*** _sigmavirus24 is now known as sigmavirus17:36
*** sigmavirus has quit IRC17:37
*** sigmavirus has joined #openstack-infra17:37
*** fbo has joined #openstack-infra17:37
dstufftjlk: fungi mordred Ok Ernest is going to follow up on that for me, likely this evening17:37
*** lbragstad has joined #openstack-infra17:37
*** jpmaxman has joined #openstack-infra17:37
*** pilgrimstack has quit IRC17:38
jlkmisfire?17:38
*** nikhil has joined #openstack-infra17:38
*** sslypushenko_ has joined #openstack-infra17:38
fungijlk: i think he meant to tab-complete jlvillal =17:39
*** SpamapS has quit IRC17:39
*** SpamapS has joined #openstack-infra17:39
*** vsaienko has quit IRC17:39
pabelangeradam_g: left a comment on 386783, otherwise I think we are good17:39
fungidstufft: thanks for checking it out17:39
*** johnsom has joined #openstack-infra17:39
*** patrickeast has joined #openstack-infra17:39
amrithmordred, yt?17:40
*** kgalanov has joined #openstack-infra17:40
*** boris-42 has joined #openstack-infra17:40
*** edwarnicke has joined #openstack-infra17:40
*** asingh_ has joined #openstack-infra17:40
*** raddaoui has joined #openstack-infra17:40
*** scottda has joined #openstack-infra17:40
*** tchaypo has joined #openstack-infra17:41
*** ggherdov- has joined #openstack-infra17:41
*** zz_dimtruck is now known as dimtruck17:41
*** csatari has joined #openstack-infra17:41
*** aimeeu has joined #openstack-infra17:41
*** wcriswell has joined #openstack-infra17:41
*** igorbelikov has joined #openstack-infra17:41
*** nt has joined #openstack-infra17:41
*** DuncanT has joined #openstack-infra17:41
*** TheJulia has joined #openstack-infra17:41
fungittx: catching up on the tc meeting log from last week, i've subscribed to the openstack-tc ml now17:42
*** geekinutah has joined #openstack-infra17:42
*** lindsayk has joined #openstack-infra17:42
*** johnsom has quit IRC17:42
*** pcrews has quit IRC17:42
*** iremizov has joined #openstack-infra17:42
*** asaprykin has joined #openstack-infra17:42
*** jaranovich_ has joined #openstack-infra17:42
*** aglarendil has joined #openstack-infra17:42
*** mgkwill has joined #openstack-infra17:42
pleia2fungi: oh yes, congrats :)17:43
*** auggy has joined #openstack-infra17:43
*** betherly has joined #openstack-infra17:43
*** skath has joined #openstack-infra17:43
*** johnsom has joined #openstack-infra17:43
*** dham1 has joined #openstack-infra17:43
*** ashtokolov has joined #openstack-infra17:43
*** annegentle has joined #openstack-infra17:43
fungipleia2: thanks. there were lots of other great candidates running, so i'm pretty sure i wouldn't have been disappointed with the outcome even if i wasn't in the final set17:44
*** cargonza has joined #openstack-infra17:44
*** michaelxin has joined #openstack-infra17:44
clarkb#status log logstash worker daemons restarted as they have all deadlocked. Proper fix in https://review.openstack.org/38812217:44
openstackstatusclarkb: finished logging17:44
clarkbok I think logstash should be happier now and will be extra happy if we update to ^17:44
*** ewindisch has joined #openstack-infra17:45
*** csatari has quit IRC17:45
*** Ng has joined #openstack-infra17:45
*** zaro has joined #openstack-infra17:45
*** fkautz has joined #openstack-infra17:45
*** sdague has joined #openstack-infra17:45
clarkbpleia2: have we double checked the image type for enter is qcow2?17:45
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: Fixed tempest regex  https://review.openstack.org/38807617:45
clarkbpleia2: I didn't end up checking that. I did check and it is another cloud that gives you wide open security groups by default \o/17:45
*** warpc has joined #openstack-infra17:45
*** mattfarina has joined #openstack-infra17:46
openstackgerritSaad Zaher proposed openstack-infra/project-config: Freezer: Fixed tempest regex  https://review.openstack.org/38807617:46
pleia2clarkb: oh, I'll doublecheck17:46
pleia2| disk_format      | qcow2                                                |17:47
pleia2yeah, we're good17:47
clarkbperfect17:47
*** apetrich has joined #openstack-infra17:49
*** amitgandhinz has quit IRC17:49
jesusaurclarkb: wow, is that whats happening? ive been trying to figure out why my log processors randomly hang for quite a while17:49
clarkbjesusaur: based on my reading of the logs and the state of the processors I think so17:50
pabelangerpleia2: clarkb: are we skipping multi-node for a reason on 388149?17:50
clarkbpabelanger: we have skipped multinode in the past because it takes more testing ot make sure the overlay network all works happily17:50
clarkbpabelanger: so yes, trying to reduce overhead for now17:50
*** adrian_otto1 has joined #openstack-infra17:51
pabelangerokay cool17:51
pabelanger+317:51
pleia2ty17:51
pabelangerso many image uploads17:51
jesusaurclarkb: would you mind also proposing a similar change to the log_processor project since we still havent taken the time to switch over?17:52
*** dprince has quit IRC17:52
*** adrian_otto has quit IRC17:52
clarkbjesusaur: sure17:53
*** kzaitsev_mb has quit IRC17:55
openstackgerritClark Boylan proposed openstack-infra/log_processor: Handle errors in previously working filters  https://review.openstack.org/38815317:55
clarkbjesusaur: ^17:55
*** gyee has joined #openstack-infra17:57
zaro clarkb, fungi, pleia2 : i have patches up for setting up a full dev CI environment (gerrit, zuul, and nodepool) basically we want to similate gerrit, zuul, nodepool integration.  The changes add a zuul-launcher and nodepool to zuul-dev.o.o.  This is part of the upgrading gerrit work.  i'm wondering whether it would be appropriate to make this a priority17:57
zaroeffort?17:57
openstackgerritMerged openstack-infra/project-config: Add entercloud regions to Nodepool  https://review.openstack.org/38814917:59
vinaypotluriHi all, I have a question related to project creation. In this section "http://docs.openstack.org/infra/manual/creators.html#add-the-project-to-the-master-projects-list"   we can specify the upstream if there is an existing repo using "upstream: https://github.com/awesumsauce/<projectname>.git" but what if we do not have one? How are we creating a new repo here inside www.github.com/openstack/projectname.git17:59
jesusaurclarkb: awesome, thanks17:59
*** nadya has joined #openstack-infra17:59
*** kzaitsev_mb has joined #openstack-infra18:00
*** yamahata has joined #openstack-infra18:00
AJaegervinaypotluri: if you have no upstream, don't specify one and an empty will get crated18:00
AJaegervinaypotluri: github is just a mirror and can go away anytime, you create a repo on git.openstack.org...18:01
vinaypotluriokay18:01
fungizaro: priority efforts generally have at least a brief spec associated with them (could probably just copy from the last gerrit upgrade spec for the most part) but sure, seems reasonable. we want to make sure the upgrade happens as early in the cycle as possible so as to minimize disruption18:02
vinaypotluriAJaeger: so the repo will be named exactly the way its created on launchpad ?  https://launchpad.net/syntribos-openstack-templates18:02
AJaegerno, it will be named exactly how you set it up. You give it a name in gerrit/projects.yaml18:02
*** nadya has quit IRC18:03
vinaypotluriAJaeger: got it. Thank You :)18:03
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: simple-init: Remove SUSE interfaces  https://review.openstack.org/38815818:03
*** mriedem has quit IRC18:04
*** psilvad has quit IRC18:04
*** mriedem has joined #openstack-infra18:05
*** mriedem has quit IRC18:05
vinaypotluriAJaeger: also, Can i edit jenkins, zuul & other jobs anytime for the created project(s) from project-config later ?18:06
clarkbpleia2: re os-xenapi-core we typicallu dont give ownership of a core group to one individual. They are self owmed to remove the spof. Probablu not an issue her but I wonder why BobBall_AWOL needs that18:06
hamzyOkay, I'm reading http://docs.openstack.org/infra/manual/creators.html and I don't see how I can create a git repository for MoltenIron (MI).  MI already exists as a subdirectory in the third party tools repo. https://git.openstack.org/cgit/openstack/third-party-ci-tools/tree/nodepool/molteniron I would like to create a new MoltenIron repo from that subdir.18:06
AJaegervinaypotluri: you can edit the setup in project-config later as well, just send another change in.18:07
pleia2clarkb: oh, I actually just added him as the first member, not owner18:07
*** dprince has joined #openstack-infra18:07
pleia2I should reply18:07
JayFhamzy: one sorta shortcut-easy way is to import that dir into a new repository somewhere else (like github), then just have infra import that repository18:07
*** Apoorva has quit IRC18:07
mgagnehamzy: better create a dummy repository already prepared and import from it?18:07
vinaypotluriAJaeger: cool...   :)18:07
JayFhamzy: then you can use "normal" git tooling to import into it18:07
*** Apoorva has joined #openstack-infra18:07
*** mriedem has joined #openstack-infra18:08
hamzyJayF, mgagne, I'll try that and create a dummy repo on github18:08
JayFhamzy: I'd just try, if possible (I don't know that it is, but I suspect it has to be?) to preserve git history18:09
hamzyYeah I was wondering about that18:09
*** piet_ has quit IRC18:10
JayFhamzy: https://help.github.com/articles/splitting-a-subfolder-out-into-a-new-repository/ looks like a howto for doing exactly what you need18:10
*** ssbarnea has joined #openstack-infra18:10
JayFhamzy: then you can take the "result" repository, inspect it for correctness, and if it's good, have infra import that18:10
hamzyI'm sure I'm not the only person to have that requirement... thanks!18:10
pabelangerinfra-root: I start my travels to the summit today and won't be in the EST timezone for the next 12days or so. Expect to be less active this week for infra things.18:11
*** amitgandhinz has joined #openstack-infra18:11
pleia2pabelanger: fun, safe travels :)18:12
openstackgerritMichael Johnson proposed openstack-infra/project-config: Adds a functional test job for Octavia  https://review.openstack.org/38807818:12
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate: Setup ssh-key on subnodes for Ironic  https://review.openstack.org/36483018:12
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate: Update ENABLED_SERVICE on subnode with ironic  https://review.openstack.org/36861118:12
openstackgerritMichael Johnson proposed openstack-infra/project-config: Adds a functional test job for Octavia  https://review.openstack.org/38807818:12
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate: Update localrc for ironic-multinode case  https://review.openstack.org/35279018:12
AJaegersafe travels, pabelanger !18:13
*** pvaneck has joined #openstack-infra18:15
*** ssbarnea has quit IRC18:16
*** amitgandhinz has quit IRC18:16
*** pilgrimstack has joined #openstack-infra18:16
*** amitgandhinz has joined #openstack-infra18:17
*** pilgrimstack has quit IRC18:17
*** esikachev has quit IRC18:19
*** abregman is now known as abregman|afk18:19
*** jcoufal_ has joined #openstack-infra18:20
anteayapabelanger: I'll add my wishes for a safe journey for you to the others18:20
*** jcoufal has quit IRC18:21
pabelangerpleia2: AJaeger: anteaya: thanks18:22
fungihamzy: usually when we create new repositories split out of the tree of an existing repo, we do a fair amount of git surgery to retain the commit history of those files. i wouldn't be comfortable trying to script that without someone to look over the results and okay them prior to import18:22
odyssey4meAJaeger pabelanger I would have thought that https://review.openstack.org/388087 should have taken effect already, but it seems not?18:23
hamzyfungi, I see the following: http://paste.openstack.org/show/586256/ which seems correct to me as there are not many commits yet as this is a somewhat new project18:23
AJaegerodyssey4me: always check http://status.openstack.org/zuul/ - the "Last reconfigured:" on the bootom18:23
*** trown|lunch is now known as trown18:23
AJaegeronly if that one is later then the merge, the change is in.18:24
*** gentoo_ has joined #openstack-infra18:24
odyssey4meAJaeger ooh, that's not something I've seen before :)18:24
odyssey4mewow - last reconfigured several hours ago if I read that right18:24
odyssey4meI thought the reconfigs happen every 20 minutes?18:25
AJaegerodyssey4me: still, an hour after the change it should be in. Let me check puppet - and then ask for help ;)18:25
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Don't use ssh-keygen -A for init scripts  https://review.openstack.org/37898518:25
*** tiswanso has quit IRC18:26
fungihamzy: sure, just commenting on why this (seemingly) common importing use case isn't something we've automated18:26
fungimost of the time the history being split out is not so trivial as your example18:26
electrofelixjust hit something very weird with http://logs.openstack.org/26/387426/3/check/gate-git-upstream-python35-nv/fdb4ce3/console.html, in that the failure is caused by not capturing any of the python logging info or debug messages during the test and I've no idea why that would suddenly start occurring18:26
*** tiswanso has joined #openstack-infra18:26
*** jkilpatr has quit IRC18:26
pabelangerAJaeger: odyssey4me: puppet apply is running now, it should happen shortly18:26
hamzyyeah I can understand that18:26
AJaegerpabelanger: according to http://puppetboard.openstack.org/, there were no puppet runs for some time and looking at http://puppetboard.openstack.org/node/zuul.openstack.org none since 12th of October - which is wrong.18:27
odyssey4methanks pabelanger :)18:27
AJaegerthanks for the info, pabelanger18:27
*** tiswanso has quit IRC18:27
med_Those getting opportunities to look for new opportunities, my heart does indeed go out to you.18:27
*** rossella_s has quit IRC18:28
electrofelixI presume nothing has changed in the python env's used for testing?18:28
*** adrian_otto1 has quit IRC18:28
*** rossella_s has joined #openstack-infra18:28
*** electrofelix has quit IRC18:28
*** mtanino has joined #openstack-infra18:29
openstackgerritVinay Potluri proposed openstack-infra/project-config: Adding a new project syntribos-openstack-templates  https://review.openstack.org/38817118:30
AJaegerarmax, dougwig , could you review project config changes as neutron infra liasons, please?18:30
armaxAJaeger: they are on my stack18:31
hamzyso JayF, mgagne, fungi, I've created https://github.com/hamzy/TempMoltenIron and it does have those five commits...  What would the next step be in getting an openstack/molteniron git repository?18:31
JayFhamzy: I think you just have to make a commit somewhere, let me find it18:31
mgagnehamzy: http://docs.openstack.org/infra/manual/creators.html#adding-the-project-to-the-ci-system make sure you use your repo in the upstream property so it's imported.18:32
pabelangerAJaeger: odyssey4me: actually, it should have landed already. It would only have updated zuul-launchers18:32
pabelangers/updated/reloaded18:33
pabelangerwhich just happened18:33
odyssey4mepabelanger thanks will recheck now18:33
hamzyJayF, mgagne, okay thanks... it was not obvious enough to me reading that document that the repo would be created after that step...18:33
AJaegerpabelanger: looking at http://status.openstack.org/zuul/, there was no reload18:33
AJaegerpabelanger: last reload was 4 hours ago18:33
JayFhamzy: yep, that's why I suggested github explicitly -- it gives you a nice public repo to feed into that upstream: parameter18:34
pabelangerAJaeger: right, zuul.o.o didn't reload. odyssey4me change was only the JJB update18:34
AJaegerpabelanger: Ah, got it finally!18:34
pabelangerlooks like we had some ssh timeouts too, so that could also explain the delay18:34
AJaegerhamzy: the repo will be created after the change to project-config is approved by our scripts18:35
*** amoralej is now known as amoralej|off18:36
*** e0ne has joined #openstack-infra18:36
*** kzaitsev_mb has quit IRC18:36
AJaegerodyssey4me: first jobs are passing..18:37
*** ihrachys has joined #openstack-infra18:37
dougwigAJaeger, armax: on it18:38
AJaegerthanks, dougwig18:38
*** e0ne has quit IRC18:39
*** jkilpatr has joined #openstack-infra18:39
*** nadya has joined #openstack-infra18:39
AJaegermordred: your change https://review.openstack.org/387990 for devstack passed tests. Do we have devstack cores to +2A around?18:39
*** e0ne has joined #openstack-infra18:39
*** e0ne has quit IRC18:41
*** mriedem has quit IRC18:42
*** eranrom has quit IRC18:43
*** claudiub has joined #openstack-infra18:44
*** lykinsbd has joined #openstack-infra18:44
*** nadya has quit IRC18:44
*** edtubill has joined #openstack-infra18:45
rcarrillocruzsorry folks, i'll pass today's meeting, my wife has evening shift today and i'm about to feed my son18:45
*** lykinsbd_ has joined #openstack-infra18:46
fungithanks for the heads up, rcarrillocruz!18:46
pleia2rcarrillocruz: have a good evening18:46
*** flaper87 has joined #openstack-infra18:46
*** flaper87 has quit IRC18:46
*** flaper87 has joined #openstack-infra18:46
rcarrillocruzfungi: as a note, cacti is back18:46
*** eranrom has joined #openstack-infra18:46
rcarrillocruzi worked with jeblair last night18:47
fungircarrillocruz: i saw some of the discussion from last week's meeting. glad you were able to work out the finer details thereafter18:47
fungifor reference, the comment in the meeting about needing to do something extra when detaching cinder volumes, we've sometimes seen detach hang because it (nova?) thinks the volume is still in use even when completely unmounted18:48
*** thiagolib has quit IRC18:48
fungithough halting the server before attempting to detach seems to always work18:48
*** haaj has quit IRC18:48
openstackgerritChristian Berendt proposed openstack-infra/project-config: Remove kolla bandit gate job  https://review.openstack.org/38790918:48
*** lykinsbd has quit IRC18:49
*** rvba has joined #openstack-infra18:50
*** rvba has quit IRC18:50
*** rvba has joined #openstack-infra18:50
*** e0ne has joined #openstack-infra18:51
*** eharney has quit IRC18:52
*** ihrachys has quit IRC18:52
AJaegerjeblair: did you already push a change our for the zuul-cloner/merger problem in periodic jobs that hits the translation-jobs?18:54
*** dtardivel has quit IRC18:54
*** jcoufal_ has quit IRC18:55
*** electrofelix has joined #openstack-infra18:56
*** kgiusti has quit IRC18:57
jeblairAJaeger: no, i'm on my 3rd attempt at a solution; i think i understand what needs to happen now :)18:57
*** jcoufal has joined #openstack-infra18:57
AJaegerjeblair: That tricky? ;( Thanks for tackling!18:57
jeblairi had to let my subconscious solve it overnight18:58
openstackgerritMerged openstack-infra/storyboard: Add updated_since to stories api  https://review.openstack.org/38712818:58
AJaegerjeblair: that's a nice trick that sometimes works wonder ;)18:58
jeblairAJaeger: yeah.  it's way smarter than i am.18:58
AJaeger:)18:59
fungiit's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour19:00
openstackgerritMerged openstack-infra/project-config: networking-odl: add bitrot job for stable branches  https://review.openstack.org/38518119:01
openstackgerritMerged openstack-infra/project-config: networking-odl: add neutron-lib-master periodic job  https://review.openstack.org/38518219:01
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: We no longer need to copy old instack.qcow2 on image promote  https://review.openstack.org/38818019:02
*** kgiusti has joined #openstack-infra19:03
BobBall_AWOLclarkb / pleia219:10
BobBall_AWOLclarkb / pleia2: I don't want to be a spof - I'm happy to go with whatever is best practice - I just want the ability to add people to os-xenapi-core and os-xenapi-ci :) I thought that was how.19:11
pleia2BobBall_AWOL: great, what I did allows you to do that :)19:12
*** jcoufal has quit IRC19:12
BobBall_AWOLAwesome - many thanks :)19:12
* BobBall_AWOL returns to being AWOL.19:12
*** jcoufal has joined #openstack-infra19:13
openstackgerritNate Johnston proposed openstack-infra/project-config: Add grenade configuration for neutron-fwaas  https://review.openstack.org/38507619:14
*** eharney has joined #openstack-infra19:14
*** psilvad has joined #openstack-infra19:17
openstackgerritKarthik Prabhu Vinod proposed openstack-infra/project-config: Adding two other combinations of cinder multinode grenade job  https://review.openstack.org/38483619:17
*** baoli has quit IRC19:18
*** cardeois has quit IRC19:19
*** gentoo_ has quit IRC19:19
openstackgerritMonty Taylor proposed openstack-infra/shade: Document and be more explicit in normalization  https://review.openstack.org/38790519:19
openstackgerritMonty Taylor proposed openstack-infra/shade: Add strict mode for trimming out non-API data  https://review.openstack.org/38818419:19
*** e0ne has quit IRC19:20
*** e0ne has joined #openstack-infra19:21
openstackgerritMerged openstack-infra/devstack-gate: Re-add dib repos to conditionally if heat is being used  https://review.openstack.org/38798819:22
*** ssbarnea has joined #openstack-infra19:22
*** electrofelix has quit IRC19:24
*** ociuhandu has joined #openstack-infra19:24
*** ssbarnea_ has joined #openstack-infra19:25
*** kaisers_ has quit IRC19:26
*** ssbarnea has quit IRC19:26
*** cardeois has joined #openstack-infra19:27
openstackgerritNate Johnston proposed openstack-infra/project-config: Add grenade configuration for neutron-fwaas  https://review.openstack.org/38507619:28
*** asselin has joined #openstack-infra19:28
*** asselin_ has quit IRC19:29
*** mriedem has joined #openstack-infra19:30
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818619:31
*** lucasagomes is now known as lucas-afk19:32
*** fernnest__ has quit IRC19:33
*** fernnest_ has quit IRC19:33
*** pcrews has joined #openstack-infra19:33
*** baoli has joined #openstack-infra19:33
*** fernnest_ has joined #openstack-infra19:34
ttxfungi: can't find you... which address did you use ?19:34
*** fernnest has joined #openstack-infra19:34
*** ssbarnea_ has quit IRC19:34
*** Goneri has quit IRC19:38
*** kaisers_ has joined #openstack-infra19:38
*** cardeois_ has joined #openstack-infra19:38
*** baoli has quit IRC19:38
*** chandankumar has quit IRC19:39
*** abregman|afk is now known as abregman19:39
*** vsaienko has joined #openstack-infra19:39
*** cardeois has quit IRC19:40
*** baoli has joined #openstack-infra19:41
*** cardeois has joined #openstack-infra19:41
*** krtaylor has quit IRC19:41
*** ssbarnea has joined #openstack-infra19:41
*** panda|bbl is now known as panda19:42
*** cardeois_ has quit IRC19:43
vinaypotluriAJaeger: Hi Andreas, Is there a ref guide i can refer to for the jobs/checks i can include in zuul configuration?19:43
vinaypotluriacc. http://docs.openstack.org/infra/manual/creators.html#basic-jenkins-jobs19:44
*** mmedvede has quit IRC19:44
AJaegervinaypotluri: not really, check what we have...19:44
*** baoli_ has joined #openstack-infra19:45
vinaypotluriokay AJaeger19:45
*** baoli has quit IRC19:45
*** baoli has joined #openstack-infra19:46
*** ssbarnea has quit IRC19:46
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818619:46
*** baoli_ has quit IRC19:46
*** ssbarnea has joined #openstack-infra19:46
*** baoli_ has joined #openstack-infra19:47
*** stream10 has joined #openstack-infra19:47
openstackgerritMerged openstack-infra/tripleo-ci: Source deploy.env in tripleo.sh  https://review.openstack.org/38775019:47
*** jcoufal has quit IRC19:47
openstackgerritKhai Do proposed openstack-infra/infra-specs: Gerrit upgrade  https://review.openstack.org/38819019:48
*** petems has quit IRC19:48
*** jcoufal has joined #openstack-infra19:48
AJaegermriedem, sdague : Could you review https://review.openstack.org/#/q/I28e157e633fe306a5939b79e38dd656b9a9ccf0b,n,z please? Those are needed for https://review.openstack.org/#/c/382232/19:49
*** petems has joined #openstack-infra19:49
*** dirk has quit IRC19:50
mriedemAJaeger: no one answered my question in https://review.openstack.org/#/c/379198/2//COMMIT_MSG19:50
*** baoli has quit IRC19:51
*** ssbarnea has quit IRC19:52
*** kun_huang has quit IRC19:52
*** kzaitsev_mb has joined #openstack-infra19:53
*** kun_huang has joined #openstack-infra19:55
AJaegermriedem: Ah - ok, let me ask...19:55
*** matt-borland has quit IRC19:55
*** nadya has joined #openstack-infra19:55
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Add janitor thread  https://review.openstack.org/38758319:56
*** ashtokolov has quit IRC19:56
*** gyee has quit IRC19:56
*** gyee has joined #openstack-infra19:56
*** Apoorva_ has joined #openstack-infra19:57
*** ramishra_ has quit IRC19:57
*** dirk has joined #openstack-infra19:57
*** ashtokolov has joined #openstack-infra19:57
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Add IPv6 coverage using existing jobs  https://review.openstack.org/38251519:58
*** rackertom has quit IRC19:58
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818619:58
*** tkelsey has quit IRC19:58
*** piet_ has joined #openstack-infra19:58
*** rackertom has joined #openstack-infra19:59
*** ramishra has joined #openstack-infra19:59
*** nadya has quit IRC19:59
openstackgerritVinay Potluri proposed openstack-infra/project-config: Adding a new project syntribos-openstack-templates  https://review.openstack.org/38817119:59
*** Apoorva has quit IRC20:00
*** baoli_ has quit IRC20:00
*** kgiusti has quit IRC20:01
*** john-davidge has quit IRC20:01
*** baoli has joined #openstack-infra20:01
*** kgiusti has joined #openstack-infra20:01
*** john-davidge has joined #openstack-infra20:01
*** burgerk has quit IRC20:01
fungittx: fungi@yuggoth.org, same as most of my other lists.openstack.org subscriptions20:01
*** e0ne has quit IRC20:01
*** esikachev has joined #openstack-infra20:02
crinklepabelanger: rcarrillocruz do you want to check in today or did pabelanger already start traveling?20:03
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818620:03
*** rfolco has quit IRC20:04
AJaegervinaypotluri: if you publish content - like your change for the new repo does, you really should test that the pbulishing works. But if it's just templates in your repo, you might not have anything to publish.20:05
*** psilvad has quit IRC20:06
AJaegerhamzy: will your repo be an official repo of ironic?20:06
*** mmedvede has joined #openstack-infra20:07
hamzyAJaeger, that is our final hope20:07
AJaegerhamzy: so, not today - then let me add some comments to your review for that20:07
vinaypotluriAJaeger: cool. In that case i can just include merge-check.20:08
*** dave-mccowan has quit IRC20:09
AJaegervinaypotluri: exactly20:09
*** Sukhdev has joined #openstack-infra20:11
ttxfungi: weird I couldn't find you,n will try again later20:11
fungittx: always possible i did it wrong, but i'll try again too in a bit20:12
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818620:14
hamzythanks AJaeger20:14
*** Sukhdev has quit IRC20:15
*** askb has joined #openstack-infra20:16
*** ldnunes has quit IRC20:17
*** john-davidge has quit IRC20:18
openstackgerritVinay Potluri proposed openstack-infra/project-config: Adding a new project syntribos-openstack-templates  https://review.openstack.org/38817120:18
openstackgerritMichael Johnson proposed openstack-infra/project-config: Consolidate neutron-lbaas API tests  https://review.openstack.org/38624520:18
*** ramishra has quit IRC20:20
openstackgerritCedric Brandily proposed openstack/os-client-config: Clarify how to set SSL settings  https://review.openstack.org/38819920:20
*** mriedem has quit IRC20:21
*** ramishra has joined #openstack-infra20:22
openstackgerritCedric Brandily proposed openstack/os-client-config: Clarify how to set SSL settings  https://review.openstack.org/38819920:23
*** mriedem has joined #openstack-infra20:24
*** claudiub has quit IRC20:26
AJaegermriedem: dirk has updated the hash in the change20:26
*** Goneri has joined #openstack-infra20:27
pabelangercrinkle: I've started travelling, but should be able to check in for tomorrow20:27
openstackgerritKhai Do proposed openstack-infra/infra-specs: add gerrit-upgrade to priority efforts query  https://review.openstack.org/38820220:28
crinklepabelanger: mmk works for me20:28
vinaypotluricraige: I saw your comments on my patch but not sure what hoop-jobs does https://review.openstack.org/#/c/388171/3/zuul/layout.yaml ? Can you please point me to any reference link?20:28
hamzyhrm... I don't understand the irc error in http://logs.openstack.org/86/388186/5/check/gate-project-config-irc-access/5d72224/console.html20:29
clarkbhamzy: I think that means you haven't configured access for our bot properly20:31
clarkbhamzy: the docs should cover the list of perms the bot needs20:31
*** jheroux has joined #openstack-infra20:32
openstackgerritVinay Potluri proposed openstack-infra/project-config: Adding a new project syntribos-openstack-templates  https://review.openstack.org/38817120:32
hamzyclarkb, Under http://docs.openstack.org/infra/manual/creators.html#configure-gerritbot-to-announce-changes ? I was just trying to add molteniron to the ironic irc channel...20:32
*** kgiusti has left #openstack-infra20:34
AJaegerhamzy: the order of entries needs to be sorted alphabetical20:36
*** stream10 has quit IRC20:36
AJaegerhamzy: check http://logs.openstack.org/86/388186/5/check/gate-project-config-irc-access/5d72224/console.html#_2016-10-18_20_19_21_056411 again...20:37
hamzyah, the + and - of the same name20:37
*** claudiub has joined #openstack-infra20:37
AJaegerhamzy: a diff ;)20:37
openstackgerritJoshua Harlow proposed openstack-infra/project-config: Refactor run-tox-with-oslo-master.sh  https://review.openstack.org/38768120:38
openstackgerritMark Hamzy proposed openstack-infra/project-config: Add new project called molteniron  https://review.openstack.org/38818620:39
clarkbAJaeger: thanks20:40
*** _ari_ has quit IRC20:41
*** _ari_ has joined #openstack-infra20:41
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: DO NOT MERGE fake periodic job to update cache to Ocata  https://review.openstack.org/38252120:42
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add small message to explain check-channels-yaml.sh  https://review.openstack.org/38820620:44
AJaegerclarkb: would this make it easier? ^20:44
*** jkilpatr has quit IRC20:44
*** trown is now known as trown|outtypewww20:46
hamzyyeah that would help newbies like myself20:47
vinaypotluriAJaeger: May I know what noop-jobs is for ?  is there any doc ref i can refer to ?20:48
*** mfedosin has quit IRC20:48
openstackgerritVinay Potluri proposed openstack-infra/project-config: Adding a new project syntribos-openstack-templates  https://review.openstack.org/38817120:51
*** jheroux has quit IRC20:51
ttxfungi: confirmed you do not appear on the Membership List for openstack-tc20:55
fungittx: i've attempted to subscribe to the openstack-tc ml again. if you still can't find it i'll try the smtp subscription approach instead of the webui20:55
fungihrm, maybe we crossed efforts there20:56
fungi"Your subscription request has been received, and will soon be acted upon. Depending on the configuration of this mailing list, your subscription request may have to be first confirmed by you via email, or approved by the list moderator. If confirmation is required, you will soon get a confirmation email which contains further instructions."20:56
fungi"OpenStack-TC list run by thierry at openstack.org"20:56
*** Apoorva_ has quit IRC20:56
ttxfungi: you should just get a confirmation email20:56
ttxI can probably force-subscribe you20:56
fungioh!20:56
*** Apoorva has joined #openstack-infra20:57
fungii bet it's going to my openstack mailing list inbox. just a sec20:57
fungii thought it was held for approval20:57
fungii haven't actually dived into my general mailing list inbox yet since getting home20:57
ttxno approval -- I only need to clear your moderation flag20:57
fungittx: thanks, subscription request confirmed now20:58
ttxhere you are.20:58
ttxmod flag cleared20:59
ttxcheers!20:59
* fungi is full of user error today20:59
fungithanks!20:59
*** raildo has quit IRC20:59
*** thorst_ has quit IRC21:00
*** esikachev has quit IRC21:01
*** tkelsey has joined #openstack-infra21:01
*** piet_ has quit IRC21:02
*** maeker has quit IRC21:02
openstackgerritMerged openstack/diskimage-builder: Add option to be able to run_functests.sh in parallel  https://review.openstack.org/34506521:03
*** jkilpatr has joined #openstack-infra21:03
cloudnullpabelanger: clarkb: fungi: are you able to pull a sample of the nodes being marked as errors in cloud1?21:05
clarkbcloudnull: I can in a few21:06
clarkb(currently working out travel thigns)21:06
* cloudnull just looking after the error rate to see if it's something on our end.21:06
*** vhosakot has quit IRC21:06
cloudnullthanks clarkb21:06
*** tkelsey has quit IRC21:06
*** ramishra has quit IRC21:07
*** gyee has quit IRC21:08
*** ramishra has joined #openstack-infra21:09
*** sdague has quit IRC21:10
*** nherciu has quit IRC21:11
*** jkilpatr has quit IRC21:11
*** eharney has quit IRC21:12
*** adriant has joined #openstack-infra21:12
*** jkilpatr has joined #openstack-infra21:12
clarkbcloudnull: http://paste.openstack.org/show/586279/ is my very initial grepping. Looks like errors waiting for ssh21:14
clarkbcloudnull: I will work on getting you uuids for each one21:14
*** lbragstad has quit IRC21:15
*** tomhambleton has quit IRC21:15
*** dirk has quit IRC21:15
*** kun_huang has quit IRC21:15
*** flaviof has quit IRC21:15
*** kuromagi has quit IRC21:15
*** kuromagi has joined #openstack-infra21:15
*** tomhambleton has joined #openstack-infra21:16
*** jbryce has quit IRC21:16
*** vaidy_ril has quit IRC21:16
*** uberjay has quit IRC21:16
*** flaviof has joined #openstack-infra21:16
*** kun_huang has joined #openstack-infra21:16
*** dolphm has quit IRC21:17
*** M-docaedo_vector has quit IRC21:17
*** knikolla has quit IRC21:17
*** jmccrory has quit IRC21:17
*** kencjohnston has quit IRC21:17
*** lVathan[m] has quit IRC21:17
*** clif_h has quit IRC21:18
cloudnullthanks clarkb21:18
*** dirk has joined #openstack-infra21:18
*** hyakuhei has quit IRC21:18
*** r-mibu has quit IRC21:19
*** thingee_ has joined #openstack-infra21:19
*** r-mibu has joined #openstack-infra21:19
clarkbcloudnull: http://paste.openstack.org/show/586281/21:19
*** dolphm has joined #openstack-infra21:19
*** hyakuhei has joined #openstack-infra21:19
*** hyakuhei has quit IRC21:19
*** hyakuhei has joined #openstack-infra21:19
*** hyakuhei has quit IRC21:19
*** hyakuhei has joined #openstack-infra21:19
*** kencjohnston has joined #openstack-infra21:19
*** thingee_ has quit IRC21:19
*** clif_h has joined #openstack-infra21:20
*** jmccrory has joined #openstack-infra21:20
*** sdake has joined #openstack-infra21:20
*** lbragstad has joined #openstack-infra21:20
*** uberjay has joined #openstack-infra21:20
*** jbryce has joined #openstack-infra21:21
*** lVathan[m] has joined #openstack-infra21:21
*** M-docaedo_vector has joined #openstack-infra21:21
*** thingee_ has joined #openstack-infra21:21
*** vaidy has joined #openstack-infra21:22
*** sdake_ has quit IRC21:22
*** Goneri has quit IRC21:25
*** ramishra has quit IRC21:27
*** abregman is now known as abregman|afk21:28
*** dprince has quit IRC21:28
jheskethMorning21:28
anteayamorning jhesketh21:29
*** maeker has joined #openstack-infra21:29
*** ramishra has joined #openstack-infra21:29
*** ociuhandu has quit IRC21:31
mriedemAJaeger: done21:31
mriedemhttps://review.openstack.org/#/q/I28e157e633fe306a5939b79e38dd656b9a9ccf0b,n,z21:31
*** maeker has quit IRC21:31
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Move job running into own class  https://review.openstack.org/38596421:32
*** jcoufal_ has joined #openstack-infra21:32
*** maeker has joined #openstack-infra21:32
*** jcoufal has quit IRC21:33
*** Jeffrey4l has quit IRC21:34
*** Jeffrey4l has joined #openstack-infra21:35
*** Rockyg has quit IRC21:37
*** jcoufal has joined #openstack-infra21:37
cloudnullclarkb: interesting, i've got nothing as to why those instnaces were unreachable21:38
*** thingee_ has quit IRC21:38
*** jcoufal_ has quit IRC21:38
cloudnullbuilding vms on the hosts where those instances were spawned is all good.21:38
cloudnulland nothing to report fault wise for those vms.21:38
* cloudnull more digging.21:39
*** ccamacho is now known as ccamacho|out21:39
cloudnullthanks for getting me that list though. much appreciate.d21:39
*** thingee_ has joined #openstack-infra21:39
*** rtheis has quit IRC21:39
*** baoli has quit IRC21:40
*** baoli has joined #openstack-infra21:40
*** cardeois_ has joined #openstack-infra21:43
*** cardeois has quit IRC21:43
*** srobert has quit IRC21:45
*** edtubill has quit IRC21:46
*** jamesdenton has quit IRC21:50
*** rfolco has joined #openstack-infra21:50
*** xyang1 has quit IRC21:55
*** annegentle has quit IRC21:57
*** jcoufal has quit IRC22:00
clarkbhrm looks like logstash workers have mostly reverted to their old stuck state again. Probably need that fix in place before things will be happy22:01
vinaypotluriHi, I'm getting "line too long" error on line 2531 even though I never added/edited that line. Any help is appreciated.   https://review.openstack.org/#/c/388174/22:01
*** annegentle has joined #openstack-infra22:02
*** kzaitsev_mb has quit IRC22:02
*** tkelsey has joined #openstack-infra22:03
*** mdrabe has quit IRC22:03
openstackgerritMerged openstack-infra/storyboard: Update story.updated_at when tasks are updated  https://review.openstack.org/38716322:03
*** Apoorva_ has joined #openstack-infra22:04
*** kzaitsev_mb has joined #openstack-infra22:05
*** askb has quit IRC22:06
*** askb has joined #openstack-infra22:06
*** abregman|afk is now known as abregman22:07
*** amitgandhinz has quit IRC22:07
*** tkelsey has quit IRC22:07
*** Apoorva has quit IRC22:07
*** dimtruck is now known as zz_dimtruck22:07
fungivinaypotluri: looks like yamllint released a new version 10 days ago, which has started identifying concerns it has with preexisting content in that repo22:12
mgagneI'm wondering about upper-constraints.txt. oslo.db in https://github.com/openstack/requirements/blob/stable/kilo/upper-constraints.txt vs https://github.com/openstack/ceilometer/blob/kilo-eol/requirements.txt#L1722:14
mgagnehow did it end up so upper-constraints.txt in g-r is greater AND outside the range in a project?22:14
clarkbmgagne: constraints is based on what you get from pip after doing a full crawl/install of the package space iirc22:15
vinaypotluriGot it. Thank you fungi22:15
fungimgagne: it's not something that gets tested for, but also i think it was after kilo that we learned the lesson that constraints should be frozen after release22:15
clarkbmgagne: and if that set works (via testing) then it is "enshrined" into constraints22:15
clarkbthe reason for this is because if you actually try to properly resolve the set you run into conflicts22:15
clarkbthere should be a non voting job that tests ^ on ever requirements change22:15
fungiconstraints was really in its infancy in kilo (and somewhat during liberty as well)22:16
mgagnetested against what? for example, with removal of namespace, oslo.db 2.0.0 can't be used22:16
clarkb(its possible that whatever caused that version of oslo.db to be used is one of the conflicts too)22:16
*** tnovacik has quit IRC22:16
clarkbmgagne: tempest on kilo iirc22:16
mgagnewas tempest testing ceilometer? :D22:16
clarkbmgagne: newton added a bunch of unittesting22:16
clarkbmgagne: probably not in that job ut I'd have to go look at what is included in the kilo tempest runs22:16
mgagnealright, don't better, won't change much to the result we have today =)22:17
*** apetrich has quit IRC22:17
mgagnejust didn't wanted to fork and maintain g-r locally22:17
clarkbmgagne: in newton we added some unittesting of various projects. Not sure if ceilometer was added22:17
fungifwiw, in mitaka and later that should be much saner22:17
mgagneok, so better have my own upper for kilo and maybe later version so I'm not victim of outside changes22:18
*** lykinsbd_ has quit IRC22:19
fungibasically, we stopped generating/approving constraints changes except for very limited bumps of our community's own libraries that had to get newer releases on those stable branches to address security vulnerabilities or other severe breakages22:19
*** apetrich has joined #openstack-infra22:19
openstackgerritJohn L. Villalovos proposed openstack-infra/project-config: Add experimental ironic grenade multitenant job  https://review.openstack.org/38823922:19
mgagnesure, just looking for a sane way to build kilo with virtualenv until I can upgrade to better22:19
fungiso constraints on more recent stable branches are pretty frozen22:19
clarkbmgagne: edit constraints to a lesser version that is acceptable and see if that works?22:20
*** mriedem has quit IRC22:20
clarkbif so you can probably also propose the change back into requirements (though at this point likely not to take patches?)22:20
mgagneyea, was adding upper-constraints.txt support to imported project but found that it doesn't work with some. =)22:20
*** gildub has joined #openstack-infra22:20
fungiupper constraints was barely used at all by anyone in kilo jobs, so it may even be entirely bogus22:21
mgagneI think I won't better at that point, gonna end up with a 8 weeks review and I will have time to upgrade to liberty or mitaka before it gets merged =)22:21
mgagnefungi: ok, won't better too much with it then =)22:22
openstackgerritgreghaynes proposed openstack/diskimage-builder: WIP nodepool element  https://review.openstack.org/38824122:22
johnsomclarkb I see you were making some projects changes, I'm seeing "diskimage-builder project was not found" in our gate runs here: http://logs.openstack.org/65/265065/18/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial-nv/ce92bdc/logs/devstacklog.txt.gz#_2016-10-18_20_29_52_60422:22
johnsomDo you have a pointer to the change that triggered this?22:22
* clarkb wonders how he keeps getting pinned for this... ianw made a change https://review.openstack.org/345818 (which I did review) to remove those things because they weren't used anywhere. Which waso nly sort of the case22:23
clarkbjohnsom: its since been fixed by readding them to devstack-gate if heat is enabled and there is a devstack change to remove the repo from devstack because it really isn't used thre22:23
johnsomHaha, sorry, just digging through changes in the last half hour and your popped up.22:24
ianwclarkb / johnsom : looks like mea cupla on that one22:24
johnsomI'm not using heat, so I suspect I still have a problem.22:25
clarkbianw: eh its ok the thing isn't actually used except that devstack wants to clone it :)22:25
*** dave-mccowan has joined #openstack-infra22:25
clarkbianw: and we fixed it while the rest of the world was blowing up due to pycparser so ya22:25
clarkbjohnsom: ah ok that would explain why it continues to break. In that case you'll want to add it explicitly to your job's PROJECT list22:25
openstackgerritgreghaynes proposed openstack/diskimage-builder: WIP nodepool element  https://review.openstack.org/38824122:26
openstackgerritMarkos Chandras proposed openstack/diskimage-builder: elements: runtime-ssh-host-keys: Add openssh-client mapping for SUSE  https://review.openstack.org/38824222:26
clarkbjohnsom: https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/octavia.yaml#n28 there I think and for the job below it22:27
*** rossella_s has quit IRC22:28
johnsomYeah, I'm going to have to do neutron-lbaas as well22:28
clarkbjohnsom: and while you are at it maybe we should add the other deps? octavia itself etc. (/me noticed the other day that we install neutron-lbaas and octavia's repos on every d-g run and was wondering why)22:28
*** rossella_s has joined #openstack-infra22:28
johnsomclarkb Because we are so awesome...  Ok, looking into this.  Do you have the patch URL so I  can reference it in the patch I'm creating.22:29
*** john-davidge has joined #openstack-infra22:30
clarkbjohnsom: https://review.openstack.org/345818 is the one that changed things for dib-utils22:30
johnsomThanks!22:30
johnsomWorking on a patch now22:30
*** cardeois_ has quit IRC22:31
clarkbianw: also fwiw I think while it may be painful to reduce that list doing so is going to make a lot of tests run much quicker. We spend 5-10 minutes just dealing with git on every d-g run22:31
clarkbif we can get that down to a minimal set everyone wins22:31
ianwclarkb: i'm all for it :)  maybe we should delete one line a week and see who complains :)22:32
clarkbianw: haha22:32
*** baoli has quit IRC22:34
*** dave-mccowan has quit IRC22:34
*** john-davidge has quit IRC22:35
clarkbjesusaur: looking at these log worker logs I think there may be a second failure mode where we wait for hours for crm114 to finish and don't timeout22:38
clarkbjesusaur: so my earlier patches may only address half the problem. Going to look and see if I can make timing out more aggressive22:38
*** rhallisey has quit IRC22:39
openstackgerritMichael Johnson proposed openstack-infra/project-config: Fix neutron-lbaas and octavia PROJECTS list  https://review.openstack.org/38824622:39
johnsomclarkb How does that look?  ^^^22:39
*** markvoelker has quit IRC22:41
clarkbjohnsom: you may also want to do it for the other jobs in neutron-lbaas.yaml but if none are currently doing diby things then they should be ok for now22:41
openstackgerritJohn L. Villalovos proposed openstack-infra/project-config: Add experimental ironic grenade multitenant job  https://review.openstack.org/38823922:41
*** gongysh has joined #openstack-infra22:41
clarkbianw: ^ can you review that as well?22:41
johnsomclarkb We have the DIB stuff limited to just the scenario tests that need the image, so it should just be those.22:42
*** edmondsw has quit IRC22:42
clarkbjohnsom: ya but eventually we do want to move lbaas and octavia and a whole list of other things out of the default d-g PROJECTS list22:43
clarkb(to reduce down to a minimal default taht reflects devstack's default which in turn will speed up jobs as they won't set up repos that they don't need22:43
clarkbso +2, but keep that in mind it would be helpful if we could do the other peice too22:44
johnsomI can do that now if we have folks to still review.  I would like to get this fixed as our scenario tests are all broken at the moment.22:44
clarkbI think taht patch is fine as is, we should get it in and fix your immediate problem22:45
*** gongysh has quit IRC22:46
clarkbianw: fungi jhesketh anteaya ^22:46
* clarkb picks on cores that were in channel recently22:46
johnsomThanks.  I need to ask dougwig what the story is with that lbaas functional test anyway.22:46
ianwit's probably a bit late US time to try removing them and merge something that adds them.  i'm trying to think of a good way to deprecate such a thing, other than just breaking stuff22:48
ianwmaybe some fancy log greps could give higher confidence about what's required for safe removal22:49
clarkbianw: ya I think just for now we want to fix the existing broken, not try to remove the additional projects22:51
jesusaurclarkb: interesting, i'll see if i can dig up some old logs, but we've taken to monitoring the age of the logfile and restarting the service if the logfile hasnt changed in an hour22:51
clarkbwe can push harder on that post summit22:51
ianwyep, cool22:51
clarkbjesusaur: as I dig more its possible that the logstash daemon may have gone out to lunch22:51
clarkbjesusaur: which results in other things backing up.22:51
clarkbjesusaur: which then leads to the timeouts (which we should still handle properly with my fix but ya more other brokenness)22:51
clarkbya I restarted lgostash daemon and now strace is telling me things are doing stuff22:52
clarkbugh thats annoying, debugging logstash is the most unfun thing to do (its jruby running in jvm)22:52
*** maeker has quit IRC22:54
johnsomWhen you guys are ready to pull neutron-lbaas and octavia, just give Doug and I a heads up so we can help anyone that comes to our channel asking what is up.22:54
*** rbrndt has quit IRC22:54
jesusauroh, hm, we restart the log_processor worker daemon... we havent had any issues with the logstash daemon22:55
clarkbjesusaur: its possible you only hit the case my patch fixes. for us that seems to be a side effect of logstash giving up, writes to the tcp connect block for forever22:56
*** gouthamr has quit IRC22:56
openstackgerritMerged openstack-infra/project-config: Fix neutron-lbaas and octavia PROJECTS list  https://review.openstack.org/38824622:57
johnsomThanks folks!22:57
*** xarses has quit IRC23:01
*** EricGonczer_ has quit IRC23:02
*** annegentle has quit IRC23:02
*** andreykurilin_ has quit IRC23:03
*** andreykurilin has joined #openstack-infra23:03
*** thorst_ has joined #openstack-infra23:04
*** yamahata has quit IRC23:04
*** yamahata has joined #openstack-infra23:05
*** tkelsey has joined #openstack-infra23:05
*** dingyichen has joined #openstack-infra23:07
pleia2all the team summit etherpads have been set up https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads#Infrastructure23:07
*** tkelsey has quit IRC23:09
*** gordc has quit IRC23:10
*** thorst_ has quit IRC23:11
*** thorst_ has joined #openstack-infra23:12
*** tpsilva has quit IRC23:14
*** abregman is now known as abregman|afk23:14
*** rajinir has quit IRC23:16
*** xarses has joined #openstack-infra23:17
*** dave-mccowan has joined #openstack-infra23:19
*** sdake has quit IRC23:20
stevebakerhey, can someone give me some guidence on what to do about this heat gate failure? http://logs.openstack.org/76/365976/2/check/gate-heat-dsvm-functional-orig-mysql-lbaasv1/5a0135b/logs/devstacklog.txt.gz#_2016-10-18_22_29_01_99223:22
*** hongbin has quit IRC23:23
clarkbstevebaker: that was a failure related to removing an unused git repo before it was actually unused. You should be able to recheck itno23:24
clarkb*it now23:24
stevebakerclarkb: ok, thanks23:24
*** thorst_ has quit IRC23:25
clarkbthere is a change to actually remove it from devstack because apparently it is unused elsewhere in the heat chain of things? just wasn't cleaned out of devstack. In the meantime we added the repos back in if running with heat enabled23:25
stevebakerheat must be close to switching over to its devstack plugin23:26
*** vsaienko has quit IRC23:35
*** aeng_afk is now known as aeng23:36
*** yamamoto has quit IRC23:38
*** maeker has joined #openstack-infra23:39
mordredstevebaker, clarkb: did that land?23:40
mordredah - ianw is my hero and approved it: https://review.openstack.org/#/c/387990/23:40
johnsomIs there a delay after merge before that change gets used?  My rechecks are failing the same way and I don't see the new PROJECTS being set.23:40
mordredjohnsom: a change to project-config?23:41
johnsomYes23:41
*** markvoelker has joined #openstack-infra23:41
mordredyah - it needs the next config-management run to happen - could be as quick as 15 minutes, or a long as an hour or so23:41
mordreddepending if something else happened23:41
johnsomAh, ok.  Thanks.  Other fallout from the DIB removal23:42
clarkb#status log restarted logstash daemons as well to get logstash pipeline moving again. Appears they all went out to lunch for some reason (logstash logs not so great but they stopped reading from the tcp connection with log workers according to strace)23:42
mordredyay dib removal!23:42
openstackstatusclarkb: finished logging23:42
*** maeker has quit IRC23:43
*** markvoelker has quit IRC23:46
*** dtp has joined #openstack-infra23:46
*** inc0 has quit IRC23:47
*** claudiub has quit IRC23:47
dtphello.  i'm trying to update a gerrit review; getting:23:47
dtp$ git review23:47
dtpfatal: ICLA contributor agreement requires current contact information.23:47
dtpPlease review your contact information:23:47
dtp  https://review.openstack.org/#/settings/contact23:47
dtpfatal: Could not read from remote repository.23:47
dtpPlease make sure you have the correct access rights23:47
dtpand the repository exists.23:47
dtp-----23:48
dtpwhen i try to update my contact info in gerrit, get:23:48
*** yamahata has quit IRC23:48
dtpCode Review - Error23:48
dtpServer Error23:48
dtpCannot store contact information23:48
dtp-----23:49
dtpwhat am i missing?23:49
clarkbdtp: http://docs.openstack.org/infra/manual/developers.html#account-setup should hopefully explain it all23:49
*** Swami has quit IRC23:51
ianwmordred: some hero ... i'm like one of the avengers who gets a bad guy but manages to destroy the whole city in the process :)23:51
mordredianw: works for me :)23:51
*** knangia has quit IRC23:52
* jeblair starts selling glass to people who need their windows replaced23:53
*** ramishra has quit IRC23:53
*** dtp has left #openstack-infra23:55
*** Apoorva_ has quit IRC23:56
*** Apoorva has joined #openstack-infra23:57
anteayadtp in future please use a paste service, like paste.openstack.org when sharing terminal output23:57
*** claudiub has joined #openstack-infra23:59
*** thorst_ has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!