Tuesday, 2013-10-22

*** gyee has quit IRC00:05
*** vipul-away is now known as vipul00:10
*** SergeyLukjanov is now known as _SergeyLukjanov00:10
* mordred is a fan of splitting00:10
* mordred thinks more of openstack should be in more repositories00:10
* mordred thinks that horizon should be split into two repos00:11
*** _SergeyLukjanov has quit IRC00:11
*** SergeyLukjanov has joined #openstack-infra00:15
*** SergeyLukjanov is now known as _SergeyLukjanov00:16
*** markmcclain has joined #openstack-infra00:19
*** matsuhashi has joined #openstack-infra00:21
*** thomasm has joined #openstack-infra00:22
*** SergeyLukjanov has joined #openstack-infra00:22
*** thomasm is now known as Guest3058000:22
SergeyLukjanovmordred, yeah, more repos is good00:23
*** rakhmerov has joined #openstack-infra00:25
*** sarob has joined #openstack-infra00:26
*** Guest30580 has quit IRC00:27
*** HenryG_ has joined #openstack-infra00:29
*** rakhmerov has quit IRC00:30
*** sarob has quit IRC00:31
*** senk has joined #openstack-infra00:31
*** HenryG has quit IRC00:31
*** zul has quit IRC00:34
*** beagles has quit IRC00:35
*** beagles has joined #openstack-infra00:35
reedmordred, you haven't even tried to keep back when writing to openstack-dev re: copyright stuff, eh? :)00:36
reedmordred, or did you? :))00:36
mordredreed: I was being very civil00:36
mordredreed: were you talking about me flaming SpamapS - or about me mentioning the ludicrous nature of our CLA00:36
reedmordred, the CLA stuff00:37
mordredit's stupid and we should stop doing it00:37
mordredI'm pretty sure everyone already knows I feel that way though :)00:37
reedwhat does the TC think of it? and how about other board members?00:37
mordreddon't know00:38
reedI think we put the CLA *and* the CCLA in the bylaws ... supermajority is needed to change those00:38
mordredyeah. I think attempting to change them will fail00:38
reedI think we'll have to live with them, consider them a technical debt00:38
mordredor remove them00:38
mordredI agree00:38
mordredit's stupid00:38
*** jaypipes has quit IRC00:38
hub_capmordred: quick Q, wrt copyright, should i do a # Copyright 2013 Rackspace Hosting or # Copyright (c) 2013 Rackspace Hosting or00:38
* reed agrees with mordred00:38
hub_capor does it matter00:39
mordredhub_cap: it does not matter00:39
hub_capless is more :)00:39
*** markmcclain has quit IRC00:39
reedhub_cap, less is more -- (C)  means nothing in law00:39
*** wenlock_ has quit IRC00:40
hub_cap++, is it a-ok for me to remove existing (c)'s or is there something morally wrong w/ editing other persons lines in copyright00:40
reeduhm ... when did ALT-gr disappear from the keyboards?00:40
reedthe right ALT key on this stupid Dell behaves just like the left ALT00:40
clarkbmordred: I agree some splitting is good but docs in its own repo and every agent is a bit much imo00:41
clarkbbecause they need to be tightly integrated00:41
mordredapparently there may be countries that are not the US that require ©00:41
mordredbut it has not been required in the US since we became berne signatories00:42
*** oubiwann_ has joined #openstack-infra00:42
hub_capok wont modify but will not add to Rackspace00:42
hub_capthx for the guidance reed and mordred, time to copyright hub_cap on all the files that i didnt touch00:43
*** reed has quit IRC00:43
clarkbmordred you could just fork openstack and accept contributions without a CLA >_>00:44
*** reed has joined #openstack-infra00:44
mordredclarkb: BWAHAHAHAAHAHA00:44
clarkbI am glad you enjoyed that00:44
reedgeez, xchat-gnome crashed... just like that00:44
Ryan_Laneso, the elasticsearch config on the wiki doesn't work :)00:45
Ryan_Lanethe wiki can't create its indexes00:46
Ryan_Lanea default config works, though00:46
*** oubiwan__ has joined #openstack-infra00:46
*** oubiwann_ has quit IRC00:46
clarkbRyan_Lane I think it has to do with the min hosts setting00:47
Ryan_Lanelet me do some testing00:47
clarkbit won't write until all hosts are present to avoid conflicts00:48
clarkbor min hosts at least00:48
clarkbwhat is core meeting starts in 10 minutes00:49
*** adalbas has quit IRC00:52
*** nosnos has joined #openstack-infra00:52
Ryan_Laneyep. found the relevant settings00:52
clarkbRyan_Lane: I think we either infer those settings from discover_hosts or have users of the elasticsearch module pass in a template with their own settings, I really don't want to parameterize every option00:53
*** sarob has joined #openstack-infra00:53
Ryan_Lanean alternative is to pass in a hash00:53
clarkband only assign the values in the hash relying on the es defaults otherwise?00:53
clarkbthat would work too00:54
Ryan_Laneand add items to the hash as we need them00:54
* Ryan_Lane hates passing in templates00:54
*** oubiwan__ has quit IRC00:57
*** nati_ueno has quit IRC00:57
*** oubiwann_ has joined #openstack-infra00:57
*** sarob has quit IRC00:57
*** zul has joined #openstack-infra00:57
mikalFor people interested in the "what is core" discussion, it is on now01:06
*** zul has quit IRC01:06
*** yaguang has quit IRC01:08
*** yolanda has quit IRC01:08
*** jerryz has quit IRC01:08
*** senk has quit IRC01:15
*** erfanian has joined #openstack-infra01:16
*** mrodden has quit IRC01:16
*** erfanian has quit IRC01:16
*** melwitt has quit IRC01:19
*** D30 has joined #openstack-infra01:25
*** dims has quit IRC01:26
*** SergeyLukjanov is now known as _SergeyLukjanov01:26
*** rakhmerov has joined #openstack-infra01:26
*** erfanian has joined #openstack-infra01:26
*** HenryG_ has quit IRC01:26
*** _SergeyLukjanov has quit IRC01:27
mordredoh. a hangout01:28
mordredmikal: what is the thing going on?01:29
mordredmikal: and why does it use non-free software01:29
*** guohliu has joined #openstack-infra01:29
*** rakhmerov has quit IRC01:30
openstackgerritRyan Lane proposed a change to openstack-infra/config: Configure elasticsearch via a hash  https://review.openstack.org/5304301:30
*** yolanda has joined #openstack-infra01:31
mikalmordred: I am merely an attendee of this hangout, which was announced on the openstack mailing list01:32
*** ryanpetrello_ has joined #openstack-infra01:32
*** ryanpetrello has quit IRC01:33
*** ryanpetrello_ is now known as ryanpetrello01:33
* mordred can't read ALL of the lists01:33
mordredI guess I'm lucky that I'm up at 2:30am in edinburgh for no good reason01:34
fungii am going to hope for a kindly google-loving hangouter to summarize later01:36
openstackgerritRyan Lane proposed a change to openstack-infra/config: Configure elasticsearch via a hash  https://review.openstack.org/5304301:36
clarkbfungi: me too01:36
fungii have enough trouble just opening google docs people send me links to without them causing browser trouble01:37
clarkbRyan_Lane: discussing the style of that change. What do you think of a template that iterates through the hash and adds key value pairs if they exist in the hashmap. Instead of special casing each option in the template?01:37
clarkbRyan_Lane: I like the verbose comments in the config files because they give me context, but I don't know how maintainable special casing each one will be01:38
*** esker has joined #openstack-infra01:38
Ryan_LaneI'm fine with that01:38
*** SergeyLukjanov has joined #openstack-infra01:38
Ryan_Laneso, just an empty file with an interation?01:38
clarkbRyan_Lane: yeah I am leaning towards that, just to make adding new options simpler01:39
Ryan_Laneare we going to use a list for the discover nodes or make people use a string?01:39
*** SergeyLukjanov has quit IRC01:39
Ryan_Lanewait. is this just yaml?01:39
clarkbRyan_Lane: it is just yaml01:39
Ryan_Lanecould just output yaml01:39
Ryan_Lanethat said....01:39
clarkbdoes puppet do that?01:39
Ryan_Lanewell, yes and no01:39
clarkbI have been using a list for discover_nodes because I use the same list for firewall settings01:39
clarkbRyan_Lane: can we convert the list to a string in puppet before passing it to the erb?01:40
clarkbI think we can with puppet-stdlib01:40
Ryan_Lanethat's necessary01:40
clarkbRyan_Lane: maybe the simple thing is to go with what you have now then deal with genericizing it later01:40
Ryan_Lanehm. that does json...01:40
Ryan_Laneruby 1.8 doesn't have ordered hashes01:41
Ryan_Laneif you output yaml/json in a template, it will change order every run01:41
Ryan_Laneand restart your daemons over and over01:41
clarkbRyan_Lane: I am fine with what you have now, we can work on making it better later01:41
Ryan_Laneyou can use ruby 1.9, but then you also need to upgrade to puppet 3 ;)01:41
clarkbwe should upgrade soon01:42
clarkbbut time01:42
Ryan_LaneI could also just iterate through a sorted hash01:42
*** SlickNik has quit IRC01:42
Ryan_Laneand detect if the variable is a list, if so, concat the list, else output the string01:43
*** SlickNik has joined #openstack-infra01:43
clarkbthat would be cool. But really I can live with the current thing as doing the iteration isn't as simple as I expected01:43
Ryan_Lane"WARNING: quoted boolean value found on line 37"01:43
*** dims has joined #openstack-infra01:43
* Ryan_Lane sighs01:43
*** mrodden has joined #openstack-infra01:44
Ryan_LaneI tried using double quotes instead of single quotes to get rid of that error01:44
Ryan_Laneis there some better way of putting true/false as a value?01:44
Ryan_Laneor should I just use true/false without quotes?01:45
clarkbRyan_Lane: single quotes didn't work?01:45
Ryan_Lanesame issue01:45
clarkbRyan_Lane: I think erb will coerce regular true/false values into the proper strings01:45
Ryan_Laneok. I'd say let's keep it like this till we upgrade, then we can just export yaml01:48
openstackgerritRyan Lane proposed a change to openstack-infra/config: Configure elasticsearch via a hash  https://review.openstack.org/5304301:48
Ryan_Lane\o/ passed that time01:51
clarkbRyan_Lane: I should be doing dinner soon. Is it ok if that doesn't get merged until tomorrow?01:52
Ryan_Laneyeah, no rush01:53
clarkbRyan_Lane: cool, I will make sure it doesn't do anything crazy to the other ES installs then push it through01:53
Ryan_Laneyep. sounds good01:53
*** dims has quit IRC01:55
*** UtahDave has joined #openstack-infra01:59
*** beagles has quit IRC02:00
*** wenlock has joined #openstack-infra02:01
*** wenlock_ has joined #openstack-infra02:13
*** SlickNik has quit IRC02:14
*** SlickNik has joined #openstack-infra02:15
*** rakhmerov has joined #openstack-infra02:26
*** rakhmerov has quit IRC02:31
*** prad has joined #openstack-infra02:36
*** prad has quit IRC02:36
*** reed has quit IRC02:38
mordredclarkb: what? dinner?02:58
*** oubiwann_ has quit IRC03:02
*** zhiyan has joined #openstack-infra03:04
*** senk has joined #openstack-infra03:10
*** xeyed4good has joined #openstack-infra03:13
*** rakhmerov has joined #openstack-infra03:27
*** rakhmerov has quit IRC03:32
*** esker has quit IRC03:45
*** UtahDave has quit IRC03:46
*** garyk has quit IRC03:48
*** rakhmerov has joined #openstack-infra03:50
*** ArxCruz has quit IRC04:10
*** yolanda has quit IRC04:13
*** wenlock has quit IRC04:16
*** UtahDave has joined #openstack-infra04:20
*** sarob has joined #openstack-infra04:22
*** guohliu has quit IRC04:27
*** xeyed4good has left #openstack-infra04:31
*** erfanian has quit IRC04:35
*** UtahDave has left #openstack-infra04:37
*** guohliu has joined #openstack-infra04:41
*** sarob has quit IRC04:45
*** sarob has joined #openstack-infra04:45
*** ArxCruz has joined #openstack-infra04:49
*** ArxCruz has quit IRC04:49
*** sarob_ has joined #openstack-infra04:50
*** sarob has quit IRC04:50
*** sarob_ has quit IRC04:51
*** sarob has joined #openstack-infra04:51
*** xchu has joined #openstack-infra04:52
*** xchu has quit IRC04:53
*** senk has quit IRC04:55
*** sarob has quit IRC04:56
*** afazekas has quit IRC04:58
*** matsuhashi has quit IRC05:01
*** matsuhashi has joined #openstack-infra05:02
*** matsuhashi has quit IRC05:02
*** matsuhashi has joined #openstack-infra05:03
*** boris-42 has joined #openstack-infra05:04
*** thomasbiege1 has joined #openstack-infra05:04
*** Loquacity has quit IRC05:06
*** SlickNik has quit IRC05:07
*** SlickNik has joined #openstack-infra05:07
*** dcramer_ has quit IRC05:11
*** nicedice_ has quit IRC05:20
*** afazekas has joined #openstack-infra05:20
*** dcramer_ has joined #openstack-infra05:24
*** vipul has quit IRC05:25
*** vipul has joined #openstack-infra05:26
*** senk has joined #openstack-infra05:26
*** thomasbiege1 has quit IRC05:27
rakhmerovguys, is there a chance you take a look at this today? https://review.openstack.org/#/c/5120505:27
*** senk has quit IRC05:33
*** matsuhashi has quit IRC05:35
*** matsuhashi has joined #openstack-infra05:36
*** matsuhashi has quit IRC05:36
*** matsuhashi has joined #openstack-infra05:37
*** thomasbiege1 has joined #openstack-infra05:37
*** ryanpetrello has quit IRC05:40
*** pblaho has joined #openstack-infra05:41
*** thomasbiege1 has quit IRC05:46
*** reed has joined #openstack-infra05:49
*** guohliu has quit IRC05:49
*** Loquacity has joined #openstack-infra05:57
*** vipul is now known as vipul-away06:00
*** sarob has joined #openstack-infra06:02
*** guohliu has joined #openstack-infra06:02
*** vipul-away is now known as vipul06:05
*** sarob has quit IRC06:07
*** thomasbiege1 has joined #openstack-infra06:11
*** SlickNik has quit IRC06:14
*** SlickNik has joined #openstack-infra06:14
*** yamahata has quit IRC06:15
*** SergeyLukjanov has joined #openstack-infra06:21
*** yamahata has joined #openstack-infra06:22
*** jcoufal has joined #openstack-infra06:25
*** xchu has joined #openstack-infra06:26
*** wenlock_ has quit IRC06:28
*** amotoki has joined #openstack-infra06:36
*** boris-42 has quit IRC06:38
*** amotoki has quit IRC06:42
*** yamahata has quit IRC06:53
*** kpepple has quit IRC06:57
*** kpepple has joined #openstack-infra06:58
*** xchu has quit IRC07:02
*** CaptTofu has quit IRC07:15
*** CaptTofu has joined #openstack-infra07:15
*** Alssi has quit IRC07:18
*** xchu has joined #openstack-infra07:19
*** xchu_ has joined #openstack-infra07:22
*** xchu__ has joined #openstack-infra07:22
*** xchu has quit IRC07:25
*** xchu has joined #openstack-infra07:25
*** xchu_ has quit IRC07:26
*** shardy_afk is now known as shardy07:27
*** xchu__ has quit IRC07:27
*** xchu has quit IRC07:27
*** xchu has joined #openstack-infra07:27
*** SergeyLukjanov is now known as _SergeyLukjanov07:37
*** _SergeyLukjanov has quit IRC07:37
*** davidhadas has quit IRC07:40
*** SergeyLukjanov has joined #openstack-infra07:46
*** katyafervent has quit IRC07:46
*** DinaBelova has joined #openstack-infra07:47
*** che-arne has quit IRC07:52
*** SergeyLukjanov is now known as _SergeyLukjanov07:55
*** boris-42 has joined #openstack-infra08:01
*** hashar has joined #openstack-infra08:01
*** senk has joined #openstack-infra08:02
*** hashar_ has joined #openstack-infra08:02
*** SergeyLukjanov has joined #openstack-infra08:03
*** jpich has joined #openstack-infra08:05
*** hashar has quit IRC08:06
*** hashar_ is now known as hashar08:06
*** dizquierdo has joined #openstack-infra08:07
*** senk has quit IRC08:08
*** SergeyLukjanov has quit IRC08:10
*** SergeyLukjanov_ has joined #openstack-infra08:10
*** yassine has joined #openstack-infra08:11
*** dachary has quit IRC08:12
*** dachary has joined #openstack-infra08:12
*** CaptTofu has quit IRC08:14
*** CaptTofu has joined #openstack-infra08:15
*** alexpilotti has joined #openstack-infra08:18
*** CaptTofu has quit IRC08:19
*** fbo_away is now known as fbo08:19
*** yamahata has joined #openstack-infra08:22
*** derekh has joined #openstack-infra08:26
*** SergeyLukjanov_ has quit IRC08:28
*** Bada has joined #openstack-infra08:32
*** SergeyLukjanov has joined #openstack-infra08:33
*** SergeyLukjanov has quit IRC08:35
*** davidhadas has joined #openstack-infra08:43
*** thomasbiege1 has quit IRC08:47
*** xeyed4good has joined #openstack-infra08:48
*** xeyed4good has quit IRC08:50
*** thomasbiege1 has joined #openstack-infra08:52
*** DinaBelova has quit IRC08:54
*** xchu has quit IRC08:56
*** thomasbiege1 has quit IRC08:58
*** thomasbiege1 has joined #openstack-infra09:02
*** senk has joined #openstack-infra09:04
*** markmc has joined #openstack-infra09:05
*** yamahata has quit IRC09:08
*** senk has quit IRC09:09
*** reed has quit IRC09:11
*** marun_ has joined #openstack-infra09:13
*** beagles has joined #openstack-infra09:14
*** marun has quit IRC09:14
*** marun_ is now known as marun09:14
*** dkliban has joined #openstack-infra09:15
*** thomasbiege1 has quit IRC09:20
*** thomasbiege has joined #openstack-infra09:34
*** D30 has quit IRC09:53
*** D30 has joined #openstack-infra09:54
*** osanchez has joined #openstack-infra09:57
*** CaptTofu has joined #openstack-infra09:59
*** senk has joined #openstack-infra10:05
*** senk has quit IRC10:09
*** thomasbiege has quit IRC10:11
*** thomasbiege has joined #openstack-infra10:13
*** thomasbiege has quit IRC10:18
*** yamahata has joined #openstack-infra10:19
*** adalbas has joined #openstack-infra10:24
*** CaptTofu has quit IRC10:25
*** CaptTofu has joined #openstack-infra10:25
*** CaptTofu has quit IRC10:30
*** CaptTofu has joined #openstack-infra10:33
*** yamahata has quit IRC10:39
*** fallenpegasus has joined #openstack-infra10:42
*** lcestari has joined #openstack-infra10:43
*** hashar has quit IRC10:46
*** yamahata has joined #openstack-infra10:51
*** fallenpegasus has quit IRC10:51
*** D30_ has joined #openstack-infra10:52
*** D30_ has quit IRC10:53
*** guohliu has quit IRC10:53
*** D30 has quit IRC10:57
*** yamahata has quit IRC10:58
*** dims has joined #openstack-infra11:00
*** CaptTofu has quit IRC11:02
*** CaptTofu has joined #openstack-infra11:02
*** jcoufal has quit IRC11:04
*** senk has joined #openstack-infra11:05
*** CaptTofu has quit IRC11:07
*** senk has quit IRC11:10
*** CaptTofu has joined #openstack-infra11:12
*** davidhadas_ has joined #openstack-infra11:20
sdagueso what needs to be done to add another git tree to the devstack nodes during prep stage?11:32
*** thomasbiege has joined #openstack-infra11:37
*** rakhmerov has quit IRC11:44
sdaguealso, unrelated once fungi or clarkb get about, the devstack bash8 job doesn't seem to be running, need some help figuring out why11:48
*** mriedem has joined #openstack-infra11:48
*** yamahata has joined #openstack-infra11:51
*** pcm_ has joined #openstack-infra11:51
*** HenryG has joined #openstack-infra11:52
*** aloga has joined #openstack-infra11:54
*** thomasbiege has quit IRC11:56
*** thomasbiege has joined #openstack-infra11:58
*** jcoufal has joined #openstack-infra12:00
*** dkranz has joined #openstack-infra12:01
*** hashar has joined #openstack-infra12:01
fungisdague: once i get some more caffeine into my thinkmeat, i'll see if i can tell why12:02
sdagueyeh, the bash8 question is my top question of the day12:02
sdaguefollowed slightly later by how we prep new repos in the devstack nodes12:03
fungiand the prep scripts nodepool runs are in our openstack-infra/config repo, but i'll get you details in a bit12:04
fungiit may grep them out of the stackrc or something, not positive12:04
*** afazekas has quit IRC12:05
*** senk has joined #openstack-infra12:06
sdagueyeh, I realized a few things get interesting here12:08
sdagueI just decided to submit a session on this to see if jeblair wants to talk about it at summit - http://summit.openstack.org/cfp/details/34012:09
*** senk has quit IRC12:10
*** che-arne has joined #openstack-infra12:12
*** yamahata has quit IRC12:13
*** sarob has joined #openstack-infra12:18
*** weshay has joined #openstack-infra12:19
*** dcramer_ has quit IRC12:19
*** dstufft has quit IRC12:21
*** afazekas has joined #openstack-infra12:21
*** dstufft has joined #openstack-infra12:22
sdagueI love an empty morning queue... makes a good time to process through the devstack changes12:25
*** sarob has quit IRC12:25
*** sarob has joined #openstack-infra12:26
*** nosnos has quit IRC12:27
*** nosnos has joined #openstack-infra12:28
*** thomasm has joined #openstack-infra12:30
*** thomasm is now known as Guest7048212:30
*** sarob has quit IRC12:30
*** rakhmerov has joined #openstack-infra12:31
*** miqui has joined #openstack-infra12:32
*** nosnos has quit IRC12:32
*** MIQUI_ has joined #openstack-infra12:34
*** Bada has quit IRC12:35
*** miqui has quit IRC12:36
*** MIQUI_ is now known as miqui12:36
*** miqui has joined #openstack-infra12:37
*** anteaya has joined #openstack-infra12:39
*** pblaho has quit IRC12:39
*** CaptTofu has quit IRC12:40
*** CaptTofu has joined #openstack-infra12:41
miqui@mordred: clarkb wanted me to run this by you, installing zuul via puppet might have an issue..after install you see this http://paste.openstack.org/show/48902/12:45
fungisdague: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n16312:45
fungisdague: that says you're only running the job on stable/folsom and stable/grizzly12:45
*** CaptTofu has quit IRC12:45
fungimiqui: i think https://review.openstack.org/51004 is his answer to that12:46
*** w_ is now known as olaph12:46
sdaguefungi: doh12:47
fungimiqui: specifically https://review.openstack.org/#/c/51004/3/modules/zuul/manifests/init.pp12:48
miqui@fungi: yup..amm there ...12:48
fungimiqui: the trick being that pip install should probably pull in your missing dependencies while setup.py install may not12:49
*** bingbu has joined #openstack-infra12:50
openstackgerritSean Dague proposed a change to openstack-infra/config: fix the inverted logic that prevented this from running  https://review.openstack.org/5310812:50
*** bingbu has quit IRC12:51
sdaguefungi: if it's possible to get that through today, that would be cool12:51
sdagueas I'd like to actually start fixing those issues, the get bash8 enforcing12:51
miqui@fungi: giving this a try now.....thanks..12:52
*** bingbu has joined #openstack-infra12:52
*** wenlock has joined #openstack-infra12:54
*** thomasbiege has quit IRC12:58
*** DennyZhang has joined #openstack-infra12:59
*** afazekas has quit IRC12:59
*** dguitarbite has joined #openstack-infra12:59
openstackgerritA change was merged to openstack-infra/config: fix the inverted logic that prevented this from running  https://review.openstack.org/5310813:00
*** dkranz has quit IRC13:02
*** dprince has joined #openstack-infra13:04
*** dguitarbite has left #openstack-infra13:05
*** alcabrera has joined #openstack-infra13:10
*** dguitarbite has joined #openstack-infra13:12
*** dguitarbite has left #openstack-infra13:14
openstackgerritDavid Caro proposed a change to openstack-infra/jenkins-job-builder: Added config options to not overwrite jobs desc  https://review.openstack.org/5208013:16
*** esker has joined #openstack-infra13:17
*** ben_duyujie has joined #openstack-infra13:21
*** matty_dubs|gone is now known as matty_dubs13:21
*** afazekas has joined #openstack-infra13:23
*** julim has joined #openstack-infra13:23
*** blamar has joined #openstack-infra13:25
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for bug 1225024  https://review.openstack.org/5296013:32
uvirtbotLaunchpad bug 1225024 in tempest "tempest.api.compute.admin.test_hosts.HostsAdminTestXML.test_list_hosts_with_zone[gate]  unexpected conductor service" [High,Confirmed] https://launchpad.net/bugs/122502413:32
*** yamahata has joined #openstack-infra13:32
openstackgerritA change was merged to openstack-infra/elastic-recheck: Remove bug 1229475 from queries.yaml  https://review.openstack.org/5296313:34
uvirtbotLaunchpad bug 1229475 in python-neutronclient "terminate_instance(): RuntimeError: Second simultaneous read on fileno 16 detected" [High,Fix committed] https://launchpad.net/bugs/122947513:34
*** yamahata has quit IRC13:40
*** yamahata has joined #openstack-infra13:41
*** thomasbiege has joined #openstack-infra13:43
*** dkranz has joined #openstack-infra13:45
*** afazekas has quit IRC13:47
*** guohliu has joined #openstack-infra13:48
*** dims has quit IRC13:54
*** dguitarbite has joined #openstack-infra13:56
*** dcramer_ has joined #openstack-infra13:56
*** thomasbiege has quit IRC13:59
*** sandywalsh has joined #openstack-infra13:59
*** thomasbiege has joined #openstack-infra13:59
*** thedodd has joined #openstack-infra14:00
*** dguitarbite has left #openstack-infra14:01
*** davidhadas__ has joined #openstack-infra14:02
*** rcleere has joined #openstack-infra14:02
*** davidhadas___ has joined #openstack-infra14:02
*** HenryG has quit IRC14:04
*** thomasbiege has quit IRC14:04
*** davidhadas_ has quit IRC14:04
*** yamahata has quit IRC14:04
*** davidhadas has quit IRC14:05
*** changbl has quit IRC14:06
*** colinmcnamara has joined #openstack-infra14:06
*** colinmcnamara has joined #openstack-infra14:06
*** dims has joined #openstack-infra14:07
*** johnthetubaguy has joined #openstack-infra14:09
*** yamahata has joined #openstack-infra14:10
*** tstevenson has joined #openstack-infra14:11
*** ben_duyujie has quit IRC14:11
*** dguitarbite has joined #openstack-infra14:13
*** HenryG has joined #openstack-infra14:13
*** luisg has joined #openstack-infra14:14
*** dguitarbite has quit IRC14:14
*** dguitarbite has joined #openstack-infra14:14
*** yamahata has quit IRC14:15
*** dguitarbite has quit IRC14:16
*** dguitarbite has joined #openstack-infra14:16
*** yamahata has joined #openstack-infra14:17
*** dkehn has quit IRC14:17
*** colinmcnamara has quit IRC14:19
*** bingbu has quit IRC14:23
*** ben_duyujie has joined #openstack-infra14:25
*** SergeyLukjanov has joined #openstack-infra14:26
miqui@clarkb: installed zuul, but was not expecting this OOTB: http://paste.openstack.org/show/48997/14:27
*** CaptTofu has joined #openstack-infra14:27
*** dkliban has quit IRC14:29
*** oubiwann_ has joined #openstack-infra14:29
*** reed has joined #openstack-infra14:29
openstackgerritVijendar Komalla proposed a change to openstack/requirements: Update python-troveclient version  https://review.openstack.org/5213714:33
openstackgerritAdrian Otto proposed a change to openstack-infra/config: Added Zuul setup to Solum Project  https://review.openstack.org/5313014:33
openstackgerritAdrian Otto proposed a change to openstack-infra/config: Adding project Solum to StackForge  https://review.openstack.org/5268314:33
*** dcramer_ has quit IRC14:34
*** dcramer_ has joined #openstack-infra14:35
*** colinmcnamara has joined #openstack-infra14:36
*** dkranz has quit IRC14:37
*** datsun180b has joined #openstack-infra14:37
*** senk has joined #openstack-infra14:37
*** xeyed4good has joined #openstack-infra14:38
*** katyafervent has joined #openstack-infra14:38
*** colinmcnamara has quit IRC14:38
*** ryanpetrello has joined #openstack-infra14:39
*** dguitarbite has quit IRC14:39
*** dkliban has joined #openstack-infra14:41
*** dguitarbite has joined #openstack-infra14:41
*** dkehn has joined #openstack-infra14:43
*** jaypipes has joined #openstack-infra14:43
*** datsun180b has quit IRC14:44
*** datsun180b has joined #openstack-infra14:44
fungimiqui: what we normally install for those two files it wants is in https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/14:45
fungisdague: to your earlier question about adding to the list of what gets cached on devstack slaves, you want to update https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/prepare_devstack.sh14:48
ryanpetrellohas something permission-wise changed w/ gerrit related to reviews?14:48
ryanpetrelloI'm getting `Permission denied (publickey).` on attempted git reviews14:49
*** tsufiev has quit IRC14:49
ryanpetrellodouble checked the key I have set in gerrit settings and it matches14:49
ryanpetrelloand the .gitreview for this project looks fine14:49
*** mestery has quit IRC14:49
sdaguefungi: thanks14:49
sdagueryanpetrello: your username wrong?14:49
ryanpetrellono, it looks right :\14:49
*** dkranz has joined #openstack-infra14:49
sdaguethat's usually the thing that gets me14:50
ryanpetrello[remote "gerrit"]14:50
ryanpetrello    url = ssh://ryanpetrello@review.openstack.org:29418/stackforge/pecan.git14:50
ryanpetrello    fetch = +refs/heads/*:refs/remotes/gerrit/*14:50
ryanpetrelloyou see anything glaringly wrong here?14:50
ryanpetrelloit won't fetch14:50
ryanpetrellookay, figured it out14:51
ryanpetrellowas being a dummy and using the wrong keyfile somewhere14:52
ryanpetrelloplease disregard :)14:52
miqui@fungi...thanks...got it..14:52
openstackgerritSean Dague proposed a change to openstack-infra/config: add pecan & wsme trees to node preparation  https://review.openstack.org/5313614:53
*** dguitarbite has quit IRC14:54
*** thomasbiege has joined #openstack-infra14:55
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Flush Gerrit caches after fixing E-mail dupes  https://review.openstack.org/5313714:55
*** thomasbiege has quit IRC14:57
*** thomasbiege has joined #openstack-infra14:58
*** matsuhashi has quit IRC14:58
*** matsuhashi has joined #openstack-infra14:59
*** cody-somerville has quit IRC15:01
*** amotoki has joined #openstack-infra15:01
*** Bada has joined #openstack-infra15:01
*** matsuhashi has quit IRC15:03
*** SergeyLukjanov is now known as _SergeyLukjanov15:05
*** _SergeyLukjanov has quit IRC15:05
*** UtahDave has joined #openstack-infra15:06
*** changbl has joined #openstack-infra15:12
*** boris-42 has quit IRC15:12
openstackgerritAdrian Otto proposed a change to openstack-infra/config: Adding project solum, with adjusted acl file  https://review.openstack.org/5313015:15
*** thomasbiege has quit IRC15:15
*** dprince has quit IRC15:20
*** SergeyLukjanov has joined #openstack-infra15:21
*** zul has joined #openstack-infra15:22
*** rnirmal has joined #openstack-infra15:26
*** thomasbiege has joined #openstack-infra15:26
*** thomasbiege has quit IRC15:27
*** thomasbiege has joined #openstack-infra15:28
*** johnthetubaguy1 has joined #openstack-infra15:31
*** SergeyLukjanov is now known as _SergeyLukjanov15:31
*** _SergeyLukjanov is now known as SergeyLukjanov15:32
*** SergeyLukjanov is now known as _SergeyLukjanov15:32
openstackgerritAnita Kuno proposed a change to openstack-infra/statusbot: Ignore all .egg files or directories.  https://review.openstack.org/5314815:32
*** _SergeyLukjanov is now known as SergeyLukjanov15:32
*** johnthetubaguy has quit IRC15:32
*** ben_duyujie has quit IRC15:34
*** osanchez has quit IRC15:34
*** osanchez has joined #openstack-infra15:35
*** CaptTofu has quit IRC15:36
*** CaptTofu has joined #openstack-infra15:37
*** johnthetubaguy1 is now known as johnthetubaguy15:38
*** HenryG has quit IRC15:38
*** HenryG has joined #openstack-infra15:38
sdaguedid gerritbot die?15:39
fungii don't think so--still see openstackgerrit in channel15:40
fungiopenstackstatus and openstack (statusbot and meetbot) seem to be absent however15:41
fungii'll have a look after the call i'm on15:41
*** CaptTofu has quit IRC15:41
*** blamar has quit IRC15:42
sdagueoh, it was just slow15:42
sdaguevery slow :)15:42
*** jcoufal has quit IRC15:43
fungiactually meetbot looks like it's still here, so just statusbot needs some help15:44
*** yaguang has joined #openstack-infra15:47
*** Guest70482 has quit IRC15:50
*** blamar has joined #openstack-infra15:51
openstackgerritKevin L. Mitchell proposed a change to openstack/requirements: Exclude iso8601 version 0.1.7  https://review.openstack.org/5315615:52
zhiyanfungi: ping15:59
fungizhiyan: hi there16:00
zhiyanfungi: hello. sorry to interrupt you, but we/glance really need infra folks help, to look about my req bump request. https://review.openstack.org/#/c/52964/16:02
*** CaptTofu has joined #openstack-infra16:02
zhiyanfungi: actually i have talked this with you yesterday. our Jenkins test are all failed, and patch are waiting now.16:03
*** davidhadas___ has quit IRC16:03
*** davidhadas__ has quit IRC16:04
fungizhiyan: you want the requirements-core reviewers: https://review.openstack.org/#/admin/groups/131,members16:04
zhiyanfungi: and I also like to make sure team's position about this, since there's a similar change, but it try to cap iso8601 0.17 version (https://review.openstack.org/#/c/53156/)16:05
fungizhiyan: i (and from what i gather the other infra core reviewers) try to stay out of requirements discussions except insofar as it affects our infrastructure or on an emergency basis when a broken dependency is getting in the way of testing/gating all of openstack and needs to be temporarily fixed16:05
*** sandywalsh has quit IRC16:05
*** danger_fo_away is now known as danger_fo16:06
*** gyee has joined #openstack-infra16:06
fungizhiyan: right now you have a downvote on that patch from one of the requirements-core members, as well as from another contributor expressing concerns over that version breaking other projects16:07
*** DennyZhang has quit IRC16:07
*** yamahata has quit IRC16:07
*** yamahata has joined #openstack-infra16:07
*** DennyZhang has joined #openstack-infra16:07
zhiyanfungi: i'm trying to get Doug Hellmann16:07
fungizhiyan: he is dhellmann-afk in irc right now, so i'm guessing he's stepped away16:08
*** Bada has quit IRC16:09
zhiyanfungi: yes, saw... how about jd__ ? Hello Julien Danjou around?16:09
*** CaptTofu has quit IRC16:11
*** CaptTofu has joined #openstack-infra16:11
annegentlereed: around?16:14
reedannegentle, in a meeting but yes16:14
fungizhiyan: i also have replied to your reply in https://launchpad.net/bugs/124250116:15
uvirtbotLaunchpad bug 1242501 in glance "Jenkins failed due to TestGlanceAPI.test_get_details_filter_changes_since" [Critical,In progress]16:15
zaroclarkb: can't find your openstack overview slides in publication project.  did you commit them?16:15
*** senk has quit IRC16:15
zhiyanfungi: cool. i agree.16:16
*** CaptTofu has quit IRC16:16
*** matty_dubs is now known as matty_dubs|lunch16:18
*** yassine has quit IRC16:20
NobodyCamhumm 500 error with https://github.com/openstack/ironic16:20
*** zul has quit IRC16:20
fungiNobodyCam: did you mean https://git.openstack.org/cgit/openstack/ironic16:21
*** mrodden has quit IRC16:21
NobodyCamlol seems I had the other bookmarked16:22
fungi(github is a proprietary service, and while we try to mirror projects to it for convenience, we can't control whether and when it breaks)16:22
pleia2bug day today16:23
pleia2fungi: do we have an icehouse milestone to point things to?16:23
*** markmc has quit IRC16:24
pleia2feels like we just did this, time sure flies :)16:24
fungipleia2: i think we need to create one. i'll see if i can figure out whether and how16:24
pleia2that's what I recall too16:24
*** yolanda has joined #openstack-infra16:26
clarkbyes it needs creating under the project itself iirc16:27
clarkbzaro yes on the overview branch16:27
fungipleia2: hrm... https://launchpad.net/openstack-ci/+milestones "Milestones belong to a series and can be created from the series page by a project owner or series release manager."16:28
fungii think that means i can't add milestones16:28
fungioh, wait, created from the series page16:29
*** jpich has quit IRC16:29
fungii should read sometimes :/16:29
*** ericw has quit IRC16:30
*** CaptTofu has joined #openstack-infra16:31
*** ericw has joined #openstack-infra16:31
fungittx: when is the icehouse release date planned?16:31
fungiadditional proof that i should read things16:32
*** adalbas has quit IRC16:34
*** rakhmerov has quit IRC16:34
*** mrodden has joined #openstack-infra16:36
fungii guessed april 17 for now16:36
fungiicehouse milestone created, pleia216:36
fungioh, right, we usually set release dates at the summit. ignore me16:39
*** anteaya has quit IRC16:40
*** adalbas has joined #openstack-infra16:42
*** yamahata has quit IRC16:43
pleia2yeah, but last time anteaya asked he said he should work on a draft, was a couple weeks ago16:43
*** derekh has quit IRC16:44
*** ericw has quit IRC16:44
*** fbo is now known as fbo_away16:44
clarkbfungi: I had the same problem when I created the havana milestone. So I went with ~6 months from grizzly release16:44
*** mestery has joined #openstack-infra16:46
*** ericw has joined #openstack-infra16:47
*** thomasbiege has quit IRC16:47
*** rnirmal has quit IRC16:48
*** amotoki has quit IRC16:50
*** hashar has left #openstack-infra16:51
*** CaptTofu has quit IRC16:53
*** DennyZha` has joined #openstack-infra16:53
*** jcoufal has joined #openstack-infra16:53
*** CaptTofu has joined #openstack-infra16:53
*** DennyZhang has quit IRC16:53
*** zul has joined #openstack-infra16:54
*** CaptTofu has quit IRC16:55
*** CaptTofu has joined #openstack-infra16:55
*** davidhadas has joined #openstack-infra16:56
*** davidhadas_ has joined #openstack-infra16:56
*** Ryan_Lane has quit IRC16:59
*** jerryz has joined #openstack-infra17:00
pleia2bug day time!17:00
pleia2and much retargeting work to do (thanks fungi for creating the new milestone)17:00
*** anteaya has joined #openstack-infra17:00
pleia2anteaya: happy bug day! https://etherpad.openstack.org/p/cibugreview-october201317:01
zhiyanjd__: ping17:01
clarkbpleia2: thank you for putting the etherpad together17:03
*** julim has quit IRC17:03
pleia2clarkb: sure17:04
*** wenlock_ has joined #openstack-infra17:04
pleia2pretty sure this was fixed https://bugs.launchpad.net/openstack-ci/+bug/1168417 confirm?17:04
fungipleia2: great work there, as always17:04
uvirtbotLaunchpad bug 1168417 in openstack-ci "https://github.com/openstack displays a low-res OpenStack logo" [Low,Triaged]17:04
pleia2it looks fixed :)17:05
fungii think mordred got antsy at one point in the middle of the night and took care of it?17:05
fungii remember some discussion around the gravatar for it17:05
pleia2yeah, something like that17:05
wenlock_hey im trying to use manage-projects on a local install for gerrit, im getting this error on a test project creation17:06
wenlock_DEBUG:manage_projects:Executing command: git --git-dir=/var/lib/jeepyb/pet-clinic/.git --work-tree=/var/lib/jeepyb/pet-clinic commit -a -mUpdate project config. --author=Gerrit Project Creator <info@cdkdev.org>17:06
pleia2doesn't seem pixely17:06
wenlock_DEBUG:manage_projects:Return code: 017:06
wenlock_DEBUG:manage_projects:Command said: [config 730ee41] Update project config.17:06
wenlock_ Author: Gerrit Project Creator <info@cdkdev.org>17:06
wenlock_ 2 files changed, 16 insertions(+), 4 deletions(-)17:06
wenlock_ create mode 100644 groups17:06
wenlock_ rewrite project.config (66%)17:06
wenlock_DEBUG:manage_projects:Executing command: git --git-dir=/var/lib/jeepyb/pet-clinic/.git --work-tree=/var/lib/jeepyb/pet-clinic push ssh://localhost:29418/pet-clinic HEAD:refs/meta/config17:06
wenlock_DEBUG:manage_projects:Return code: 117:06
wenlock_remote: Processing changes: refs: 1, done17:06
wenlock_To ssh://localhost:29418/pet-clinic17:06
wenlock_ ! [remote rejected] HEAD -> refs/meta/config (can not update the reference as a fast forward)17:06
wenlock_error: failed to push some refs to 'ssh://localhost:29418/pet-clinic'17:06
wenlock_any ideas?17:06
wenlock_sorry, for text17:06
pleia2wenlock_: we love paste.openstack.org :)17:06
wenlock_i know, my bad17:06
wenlock_i thought it would be shorter17:06
*** thomasbiege has joined #openstack-infra17:06
*** julim has joined #openstack-infra17:06
*** DennyZha` has quit IRC17:06
clarkbwenlock_: are you using latest jeepyb?17:07
wenlock_yep, i think so17:07
wenlock_is there a way to check the version?17:07
clarkbI have a hunch this is a side effect of maintaining state across runs. If changes were made in gerrit directly and not reflected to the local state then jeepyb changes may be editing on old state and have conflicts (non fast forward)17:07
clarkbmordred: ^ do you know if that is possible?17:07
clarkbwenlock_: pip freeze | grep jeepyb17:08
*** boris-42 has joined #openstack-infra17:08
wenlock_clarkb: jeepyb==39948a317:08
wenlock_clarkb: 1 thing, is the remote: for the config of that project isn't an openstack project.... it's a project on github17:09
clarkbwenlock_: yup that is latest jeepby (I confirmed by checking the commit sha)17:09
wenlock_ https://github.com/Stackato-Apps/pet-clinic.git17:10
clarkbwenlock_: I don't think that matters in this case. the command being run is git push localhost HEAD:refs/meta/config17:11
wenlock_yep, it's just trying to update the local serves project.config i think17:11
clarkbwenlock_: I think an out of band change was made to refs/meta/config and it has confused jeepyb. You can confirm by checking the git sha in the gerrit repo and the jeepyb repo17:11
wenlock_and having a hard time pushing that change into the branch17:11
*** markmcclain has joined #openstack-infra17:11
*** davidhadas has quit IRC17:12
clarkbfungi: https://bugs.launchpad.net/openstack-ci/+bug/1010621 can I bug you to update that bug with your thoughts on backing up puppetmaster? I remember them being excellent ideas17:12
uvirtbotLaunchpad bug 1010621 in openstack-ci "important servers should have backups" [Critical,In progress]17:12
fungiclarkb: you bet--on it17:12
zhiyanmarkmcclain: ping17:12
*** davidhadas_ has quit IRC17:12
*** davidhadas has joined #openstack-infra17:13
clarkbwenlock_: that error message basically means that pushing the jeepyb changes to gerrit will result in a merge rather than a fast forward which isn't allowed without the force option17:13
wenlock_is there a way to tell manage-projects to force it?17:14
*** nicedice_ has joined #openstack-infra17:14
wenlock_its a fresh empty project17:14
*** dkliban has quit IRC17:14
wenlock_so anything jeepy believes it should be should be17:14
clarkbwenlock_: are you sure it is a fresh project?17:15
wenlock_manage-projects created it in gerrit17:15
wenlock_i tried the same with a 2nd project17:15
clarkband no there isn't a way for jeepyb to force it because typically that is a bad thing, we need to sort out why it isn't a fast forward17:15
clarkbI think you will need to check the git logs for refs/meta/config in each repo17:16
*** sandywalsh has joined #openstack-infra17:16
*** anteaya has quit IRC17:17
clarkbanteaya: pleia2: https://bugs.launchpad.net/openstack-ci/+bug/1082787 any chance that can be updated with progress on the puppet dashboard stuff?17:17
uvirtbotLaunchpad bug 1082787 in openstack-ci "Migrate remaining servers from rackspace classic to rackspace nova" [High,Triaged]17:17
*** davidhadas has quit IRC17:18
pleia2clarkb: on it17:19
fungii keep meaning to ask Ryan_Lane when he's around whether he's got any recommendations on a mw plugin to simplify https://launchpad.net/bugs/123305917:20
uvirtbotLaunchpad bug 1233059 in openstack-ci "Set up governance wiki pages as read-only copies of reference repo" [High,Triaged]17:20
*** prad has joined #openstack-infra17:20
clarkbzaro: https://bugs.launchpad.net/openstack-ci/+bug/1082812 can you update that bug with info on automated publishing to maven central eg the stuff about gpg signing packages and so on17:21
uvirtbotLaunchpad bug 1082812 in openstack-ci "Move rackspace/clouddocs-maven-plugin to openstack-dev/clouddocs-maven-plugin" [High,In progress]17:21
clarkbfungi: ^ that bug may be of interest to you as well17:21
*** matty_dubs|lunch is now known as matty_dubs17:21
*** dizquierdo has left #openstack-infra17:22
*** SergeyLukjanov has quit IRC17:22
clarkbalso please retarget bugs to icehouse if we can't close them today17:23
pleia2clarkb: btw, only ~openstack-ci-core can change milestone (not me!)17:25
clarkbpleia2: ah17:26
*** thomasbiege has quit IRC17:27
*** markwash has joined #openstack-infra17:29
*** rakhmerov has joined #openstack-infra17:29
clarkbfungi: https://bugs.launchpad.net/openstack-ci/+bug/1129172 any chance we can get a tl;dr of the irc discussion?17:30
uvirtbotLaunchpad bug 1129172 in openstack-ci "devstack-gate - Unable to get pack index" [High,Incomplete]17:30
fungiclarkb: added to the bug as well, but the irc discussion was a bug state administrivia meta-discussion17:33
clarkbfungi: danke17:33
fungiparticularly that setting a bug to incomplete should be enough for us to not have to worry about it on a bug day17:33
clarkbjgriffith: https://bugs.launchpad.net/openstack-ci/+bug/1225664 is that still a problem?17:34
uvirtbotLaunchpad bug 1225664 in openstack-ci "tempest.api.volume.test_volumes_actions.VolumesActionsTestXML flakey failure" [High,Triaged]17:34
*** zul has quit IRC17:34
clarkbfungi: gotcha, should we remove it completely from this etherpad then?17:34
clarkbjgriffith: looking at http://status.openstack.org/elastic-recheck/ it appears to be17:34
*** SergeyLukjanov has joined #openstack-infra17:35
markwashany requirements core here to help out with https://review.openstack.org/#/c/53156/ ?17:35
clarkbfungi: can I also bug you to look at https://bugs.launchpad.net/openstack-ci/+bug/1229151 and change the priority if it is a low hanging fruit17:36
uvirtbotLaunchpad bug 1229151 in openstack-ci "Git replication failing for new projects" [High,Confirmed]17:36
clarkbmarkwash: so, I think consensus has been that you should deal with both17:36
markwashglance tests are broken due to api changes* in iso8601 0.1.7 release17:36
clarkbmarkwash: rather than cap iso860117:37
markwashclarkb: that change doesn't involve capping, just excluding 0.1.717:37
markwashclarkb: but point taken, let me look again17:37
clarkbmarkwash: though looking at it more closely this is related to py3k and not the date parsing17:38
clarkbmarkwash: what py3k tests fail? I thought novaclient was the only project with passing py3k tests17:38
markwashclarkb: alas, I'll have to defer to vek for that17:39
zhiyanclarkb: deal both will cause glance-{api, registry} provide inconsistent api result. when deployer using iso8601 0.1.7 then the "changes-since" filter param for the image-list API will support yyyy-mm-dd param (no time part), but when deployer using 0.1.4 then that same request will raise an exception. This's unreasonable to me (and end user IMO).17:39
clarkboh hey it is novaclient that failed17:39
clarkbzhiyan: I think the suggestion was deal with both in such a way that the api would be consistent17:39
markwashzhiyan: we may have to backport a forward-compatible fixup17:39
clarkbhowever, now that I see novaclient is affected I am fine with excluding 1.7 as well17:39
clarkbsince that is an install problem it is a bit harder to work aroubnd17:40
markwashI guess it looks like we need to do both, because I bet iso8601 will fix the unicode issue soon and then we'll have the same problem with 0.1.8 as was pointed out17:40
clarkbmarkwash: right, so glance will need to deal with both eventually, but I think we can exclude 1.7 now sanel17:41
*** hartsocks has joined #openstack-infra17:41
markwashit would be a nice way to buy a few days so that this generally low significance problem isn't a merge blocker :-)17:41
clarkboh and 1.8 was released an hour ago17:41
markwashohlol nm17:41
zhiyanclarkb: so can we excluding 0.1.7 first for the unicode issue fixing, then to make glance forward-compatible fixup?17:41
clarkbyeah, +2 from me17:41
markwashzhiyan: I think since 0.1.8 is out now, we have to change glance immediately anyway17:42
markwashclarkb: thanks for the help17:42
clarkbmarkwash: you have until our mirro updates which is ~4pm PST if nothing gets to it first17:42
zhiyanmarkwash: probably we can ask glance using >=0.1.8 ?17:42
zhiyanclarkb: thanks!17:42
markwashzhiyan: no, I think we need to change the registry code17:43
*** davidhadas has joined #openstack-infra17:43
markwashzhiyan: we need to be compatible with both 0.1.4 and 0.1.817:43
zhiyanmarkwash: wen can talk this in glance room later.17:44
jgriffithclarkb: yes, however as I recall it's an issue on the glance-client side of things, or possibly swift17:44
clarkbjgriffith: should I remove openstack-ci from the bug then?17:45
jgriffithclarkb: IMO yes17:45
clarkbjgriffith: awesome will do17:45
jgriffithclarkb: if I'm wrong it can always be added back in17:45
*** dhellmann-afk is now known as dhellmann17:45
jgriffithclarkb: I was actually going to remove cidner as well17:45
fungiwhat's our milestone targeting threshold for bug severity? medium or high?17:50
clarkbfungi: I have been targetting mediums17:51
clarkbfungi: if they were previously targetted17:52
*** zul has joined #openstack-infra17:53
*** atiwari has joined #openstack-infra17:54
*** melwitt has joined #openstack-infra17:54
clarkbhmm we lost anteaya17:55
*** Ryan_Lane has joined #openstack-infra17:55
clarkbpleia2: fwiw https://bugs.launchpad.net/openstack-ci/+bug/1021696 may be fixed by a new gerritlib release, lifeless made the gerritlib thread a daemon thread. I will update the bug17:55
uvirtbotLaunchpad bug 1021696 in openstack-ci "gerritbot should be interruptable" [Medium,Triaged]17:55
*** Ryan_Lane has quit IRC17:57
pleia2clarkb: nice!17:57
*** yolanda has quit IRC17:57
*** osanchez has quit IRC17:58
openstackgerritSean Dague proposed a change to openstack/requirements: Revert "Unpin iso8601 but skip bad versions"  https://review.openstack.org/5318018:00
sdaguemarkwash: so I think that's actually the change we need18:00
markwashsdague: sounds prudent to me. . mostly I just want glance to work :-)18:01
sdaguemarkwash: sure :)18:01
sdagueI was just completely surprised that cap was removed, as it took long enough to pin it :)18:01
*** anteaya has joined #openstack-infra18:01
annegentlefungi: around?18:02
* anteaya reads bug day etherpad and shakes fist at router/computer sleep config18:02
fungiannegentle: yes, call in ~25 minutes right?18:03
*** che-arne has quit IRC18:03
annegentlefungi: yep this is for something else, will PM18:03
clarkbsdague: markwash: I disagree, we are at the beginning of the cycle we should be fixing those things properly18:03
clarkbsee sqlalchemy for reasons :)18:04
markwashclarkb: Imma fix glance regardless don't you worry18:04
sdagueclarkb: so we've got 1 working known version, and 3 known to be not working versions18:04
sdagueor are we all sure that 0.1.8 works for reals this time?18:05
*** hartsocks1 has joined #openstack-infra18:05
clarkbsdague: I don't think we are sure sure, but they didfix the bug that broke novaclient18:05
fungiheh, #define "works18:05
clarkbsdague: and the thing breaking glance in 1.7 will have to be worked around I think18:06
sdagueso https://review.openstack.org/#/c/53156/1/global-requirements.txt just seems crazy pants to me18:06
*** hartsocks has quit IRC18:06
*** hartsocks1 is now known as hartsocks_18:06
clarkbsdague: it is a little bit18:07
sdagueI'm happy with ==1.4.0 or >=1.8.0, but I think the growing list of "nope, doesn't work right, block it, but keep the open ended upgrade" needs to stop18:07
*** zul has quit IRC18:08
*** anteaya has quit IRC18:08
*** Ryan_Lane has joined #openstack-infra18:08
clarkbI don't like upper pins which is why ==1.4.0 bothers me, I am fine with >=1.8.0 but that may put pressure on other folks18:09
*** anteaya has joined #openstack-infra18:10
*** thomasm has joined #openstack-infra18:11
*** thomasm is now known as Guest2505118:11
fungisdague: the counterargument is from projects which have no problems with those earlier versions, versions which are present and entrenched on stable distributions, and don't want to give the impression that they don't support it18:12
sdagueI think my concern is that if in the enumerated list of releases out there more of them explicitly don't work than do, it's kind of crazy to be open ended on the requirements18:13
sdagueas it seems a high likelihood that a future release will break us18:13
sdagueanyway, if someone wants to override, so be it :)18:14
hub_capjeblair: i will likely be out of pocket today (conference) for the weekly. suffice it to say that ive done the legwork to understand what i need to do w/ the image caching but havent done it yet. if someone else is taking it over i understand, but ill get something cranked out this wk if not18:14
sdagueI didn't block it, it just seemed a bit nutty to me18:14
* LinuxJedi is with sdague. We (libra) have been burnt before by new packages breaking compatibility when using >=18:15
LinuxJedieven during minor point releases18:15
fungii agree it's nutty, but if we can't feel comfortable with any new releases of iso8601 ever again, we should reimplement or fok18:15
clarkbLinuxJedi: right, but you want that18:15
clarkbotehrwise you end up with the sqlalchemy situation18:15
clarkbits better to break occasionally then have to do crazy things 6 months later when distros change their versions18:15
LinuxJediwith something we want people to deploy it is better to not break at all18:16
*** yolanda has joined #openstack-infra18:17
LinuxJedipython-novaclient is a classic example that breaks API compatibility in almost every point release which breaks things for us (and happened during a deployment before)18:17
*** anteaya_ has joined #openstack-infra18:17
*** johnthetubaguy has quit IRC18:17
fungiwell, that's one under our control to wrangle at least, and getting backward-compat/api-compat gating in place should help tremendously18:19
LinuxJedithis is why we use == or < and custom PPAs18:19
*** anteaya has quit IRC18:19
LinuxJedifungi: that will _never_ happen :)18:20
sdagueso the thing is, in our gate, unless we >= 1.8.0, we're not testing it, because precise provides 0.1.418:20
clarkbwe must not be using volatile bits of novaclient18:20
harlowjafungi + others uploaded https://pypi.python.org/pypi/gerrit-view so that its easier to install :)18:20
LinuxJediclarkb: exceptions and debugging are the two very volatile parts18:20
clarkbsdague: we are testing with the newer versions, we know this because we keep breaking under the newer versions :)18:20
sdagueclarkb: not in devstack gate18:21
sdagueI was just looking through logs18:21
sdagueso on the unit tests nodes, maybe18:21
clarkbso unittests with virtualenvs test with newer versions, but global installs don't18:21
LinuxJedianyway, this is Openstack so has nothing to do with me, I will butt out :)18:22
clarkbI just think the only way we can avoid the sqlalchemy problem happening again is to tackle these problems head on particularly at the beginning of a cycle18:22
sdagueyeh, I agree, so let's see if 1.8.0 actually works :)18:22
openstackgerritSean Dague proposed a change to openstack/requirements: require iso8601>=0.1.8  https://review.openstack.org/5318718:24
sdagueok, let the gate decide18:25
clarkbI suddenly have an image of gladitorial combat18:27
clarkbsdague: fwiw I think glance will fail until the compat in date parsing stuff is sorted18:27
*** CaptTofu has quit IRC18:27
*** CaptTofu has joined #openstack-infra18:29
*** shardy is now known as shardy_afk18:29
*** rakhmerov has joined #openstack-infra18:30
*** SergeyLukjanov has quit IRC18:31
openstackgerritChris Jones proposed a change to openstack-infra/config: Add release config for tuskar components.  https://review.openstack.org/5319018:32
openstackgerritChris Jones proposed a change to openstack-infra/config: Add release config for tripleo-heat-templates.  https://review.openstack.org/5220618:32
*** Ryan_Lane has quit IRC18:32
*** Ryan_Lane has joined #openstack-infra18:33
*** wenlock__ has joined #openstack-infra18:33
*** rakhmerov has quit IRC18:35
*** wenlock_ has quit IRC18:36
*** markmcclain has quit IRC18:38
*** yolanda has quit IRC18:39
clarkbdstufft: can I get you to look at https://bugs.launchpad.net/openstack-ci/+bug/1150090 jog0 marked it invalid which I am leaning towards as we don't delete old releases18:39
uvirtbotLaunchpad bug 1150090 in python-swiftclient "Please do not hide released versions on pypi.python.org" [Undecided,New]18:39
clarkbdstufft: but figure you will know if there is actually an issue to be addressed there18:39
dstufftthere's no18:39
dstuffthiding only hides it in the UX18:40
*** thomasm_ has joined #openstack-infra18:40
*** melwitt has quit IRC18:41
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Add havana branch to gerritbot for #openstack-doc  https://review.openstack.org/5319218:43
*** Guest25051 has quit IRC18:43
*** locke105 has quit IRC18:44
clarkbdstufft: thanks18:46
clarkbsdague: https://bugs.launchpad.net/openstack-ci/+bug/1206073 can that be marked fix released as part of havana?18:46
uvirtbotLaunchpad bug 1206073 in openstack-ci "cinder volume fails to report in if jsonschema doesn't match what it was compiled with" [Undecided,New]18:46
*** vipul is now known as vipul-away18:46
dhellmannsdague: do we want >=0.1.8 or >=0.1.4 now?18:47
clarkbdhellmann: we are experimenting with the >-0.1.8 to see if it works18:47
dhellmannclarkb: yep, looks like those tests just passed18:48
dhellmannclarkb: but it wasn't clear if we just wanted to know that it worked, or if we want it as a min18:48
dhellmannclarkb: oh, wait, that must have been one of the other sets of tests for another changeset18:48
* dhellmann is confused18:48
openstackgerritChris Jones proposed a change to openstack-infra/config: Add release config for tuskar components.  https://review.openstack.org/5319018:49
*** melwitt has joined #openstack-infra18:50
ttxfungi: it's likely to be a bit later than April 17.18:50
clarkbdhellmann: it is confusing. I am arguing that we shouldn't pin the upper bound and sdague is arguing we should require >=0.1.8 if it works instead of excluding a bunch of broken versions (also 0.1.4 is used on our precise hosts if we allow that version during the tempest tests)18:50
clarkbdhellmann: so we are compromising on saying we should require >=0.1.818:50
*** yolanda has joined #openstack-infra18:52
*** yaguang has quit IRC18:52
*** anteaya_ is now known as anteaya18:53
anteayaare we having a meeting in 7 minutes and if yes, who is chairing?18:53
*** yaguang has joined #openstack-infra18:54
pleia2yes, don't know18:55
mgagneQuestion related to deep_format in JJB: when a value is "formatted", is the formatted value supposed to replace the old one? The problem is in this example (pipeline-next one) which does not work: http://ci.openstack.org/jenkins-job-builder/configuration.html#job-group '{pipeline-next}' is not substituted properly in the job itself.18:55
*** derekh has joined #openstack-infra18:55
*** Loquacities has joined #openstack-infra18:55
clarkbfungi or I can do it or anyone else really :)18:56
fungiannegentle: i'm going to quickly throw the doc edits topic onto our -infra meeting agenda for today, but mainly just because i want to talk about technical implementation details to support what we covered on the call18:56
anteayagreat, just wanted to tell the chair that my internet connection is unreliable this afternoon, not sure if router or modem yet18:57
clarkbmgagne: does it get substituted as {name}-upload?18:57
clarkbmgagne: or not substituted at all? we are using project specific substitutions in the openstack JJB configs so it works in some cases18:57
mgagneclarkb: yes, that's it18:57
mgagneclarkb: it's partially substituted in fact18:58
*** CaptTofu has quit IRC18:58
clarkbmgagne: yeah I think it can't do recursive replacements18:58
clarkbso {name}-upload doesn't get updated18:58
mgagneclarkb: so the provided example simply does not work? =)18:58
*** CaptTofu has joined #openstack-infra18:58
clarkbfungi: https://bugs.launchpad.net/openstack-ci/+bug/1217815 did you add that? jeblair didn't update the bug after the meeting18:58
uvirtbotLaunchpad bug 1217815 in openstack-ci "Tripleo ci service account in gerrit" [Undecided,New]18:59
clarkbpleia2: ^ maybe you know18:59
clarkbmgagne: looks like it18:59
* mgagne is a sad panda18:59
*** Loquacity has quit IRC18:59
dhellmannclarkb, sdague : if 0.1.4 works I don't see a problem with including it, but raising the lower bound makes sense, too19:00
pleia2clarkb: I don't remember19:00
fungimeetin time19:00
sdagueclarkb: so https://review.openstack.org/#/c/53187/ doesn't work19:01
sdaguedhellmann: the issue is we don't actually test 0.1.8 if we allow 0.1.4 on the devstack nodes19:01
sdaguebecause we've installed the deb19:01
dhellmannsdague: ok, so I guess we need to raise the limit19:01
*** dkliban has joined #openstack-infra19:02
sdagueso 0.1.8 isn't in our mirror yet19:03
*** CaptTofu has quit IRC19:03
*** dmakogon has joined #openstack-infra19:04
*** markmcclain has joined #openstack-infra19:04
annegentlefungi: ok sounds good19:04
annegentlefungi: they seem really interested in keeping us happy :)19:05
*** dmakogon has left #openstack-infra19:05
roaethowdy infra folk. I'm running into a 1hr timeout on a neutron test, and I don't know what is happening https://jenkins02.openstack.org/job/gate-neutron-python26/2462/19:06
annegentlefungi: I have to go run some errands but I'll be back in a while19:06
fungiannegentle: sounds good. i'll let you know if anything comes up with requires adjustment to what we talked about, however unlikely19:06
clarkbroaet: looks like the conversion from subunit to html is taking forever, infra is in a meeting now or I would look closer. You can grab the subunit log file once the test dies and try converting it locally to see if there is something blatantly wrong19:09
clarkbroaet: actually it succeeded. But the file conversion is still taking forever so we should look at that19:11
roaethttp://logs.openstack.org/42/44942/5/check/check-tempest-devstack-vm-neutron-pg-isolated/8f61c65/console.html failed19:11
*** SergeyLukjanov has joined #openstack-infra19:12
clarkbroaet: that one failed in the tests19:12
roaetbut it passed last time it ran.19:12
roaetit's very confusing19:13
clarkbroaet: right, so some of our tests (particularly the integration tests) fail on some frequency19:13
*** dstufft has quit IRC19:13
roaetclarkb: so I should just recheck no bug?19:13
clarkbroaet: because the tests tickle bugs that don't show up 100% of the time19:13
clarkbroaet: no19:13
clarkbroaet: recheck no bug is for when there is no bug associated with the need for a recheck. In this case there should be a bug because it tickled a failure somewhere in openstack. Look at http://status.openstack.org/rechecks/ and see if the bug is present and use that number19:14
*** markmcclain has quit IRC19:15
*** markmcclain has joined #openstack-infra19:15
roaetclarkb: thank you for the info, I was not aware of how to do that.19:17
clarkbroaet: the details should also be documented in the link Jenkins provides you in the gerrit comments when tests fail19:17
roaetclarkb: so it told me that I may have hit https://bugs.launchpad.net/neutron/+bug/1235486, should I recheck bug 1235486 then?19:18
uvirtbotLaunchpad bug 1235486 in neutron "Integrity violation on delete network" [Medium,In progress]19:19
uvirtbotLaunchpad bug 1235486 in neutron "Integrity violation on delete network" [Medium,In progress] https://launchpad.net/bugs/123548619:19
clarkbroaet: yes19:19
roaetWhat exactly mean then, does it test for that bug to reoccur? Or will it just run for that bug?19:20
clarkbthe idea is that we need to track this data, recheck bug XXXXX allows for us to track the data19:21
clarkbit will just rerun your tests the same way they ran the first time19:21
clarkbbut now we know we hit that bug19:21
roaetclarkb: ok. thank you, what if I disagree with jenkins? I feel like I hit another bug, should I do both?19:22
roaetAs I definitely hit https://bugs.launchpad.net/neutron/+bug/123215519:23
uvirtbotLaunchpad bug 1232155 in neutron "TypeError MagicMock not JSON serializable on local UT run" [Undecided,In progress]19:23
clarkbroaet: no don't do both, but you can use a different bug number if you like19:23
*** senk has joined #openstack-infra19:25
*** dcramer_ has quit IRC19:25
*** thomasm_ has quit IRC19:26
lifelessis fully approved but has not landed19:27
lifelesswhat should we do to make it actually land ?19:27
mgagnelifeless: it depends on https://review.openstack.org/#/c/44500/919:27
*** cody-somerville has joined #openstack-infra19:27
lifelessmgagne: ahha, I see19:28
lifelessrussellb: so do you think it would be straightforward to highlight things like that - it shows pu in our 'oldest with no -2/-1 since last nack'19:29
lifelessrussellb: but it's not actually inventory, since it's in a dependency chain19:29
*** rakhmerov has joined #openstack-infra19:30
*** gyee has quit IRC19:32
*** adalbas has quit IRC19:34
*** rakhmerov has quit IRC19:35
lifelessmeetbot seems to be ignoring things like #action and #vote in -meeting-alt19:36
clarkbI think #action is a silent response and #startvote is the command to begin a vote19:37
russellblifeless: it'd be easy to filter out stuff already marked as approved, yeah19:38
lifelessrussellb: yeah, thats an obvious thing to do19:40
russellblifeless: on it19:40
*** dkliban_ has joined #openstack-infra19:40
*** dkliban has quit IRC19:40
lifelessrussellb: I guess I'm also thinking about the stats etc, in that only the top of a dependency chain really matters in terms of getting reviews etc, because it has to land before the rest become relevant19:40
lifelessrussellb: thanks!19:40
russellbnot sure if gerrit returns the dependency info though19:41
openstackgerritRussell Bryant proposed a change to openstack-infra/reviewstats: Ignore approved changes in openreviews stats  https://review.openstack.org/5319819:43
openstackgerritA change was merged to openstack-infra/reviewstats: Ignore approved changes in openreviews stats  https://review.openstack.org/5319819:44
*** thomasm has joined #openstack-infra19:46
*** thomasm is now known as Guest5616919:47
*** dstufft has joined #openstack-infra19:48
*** weshay has quit IRC19:53
lifelessrussellb: nuts, I missed out on the change to review it :P19:53
lifelessrussellb: how am I going to get good reviewer stats ?19:54
russellblifeless: :-p19:54
russellblifeless: current stats should reflect that change19:54
russellbmy reviewstats review stats are probably awesome19:54
lifelessrussellb: brilliant19:54
*** weshay has joined #openstack-infra19:55
*** Bada has joined #openstack-infra19:55
*** SergeyLukjanov is now known as _SergeyLukjanov19:57
*** _SergeyLukjanov has quit IRC19:58
*** locke105 has joined #openstack-infra19:58
sdagueclarkb: so I was thinking about - https://review.openstack.org/#/c/49466/ this morning19:58
sdagueyou know, I think we should actually make that something which configures down from devstack-gate, so we could set it to 8 threads in d-g, and it will run that way regardless of underlying hardware19:59
clarkbsdague: you can override concurrency to testr with `testr --parallel --concurrency=8` maybe just have d-g pass that flag into tox when running the tempest tests?20:00
*** HenryG has quit IRC20:01
*** pcm_ has quit IRC20:03
mtreinishclarkb: I think we've got posargs set correctly for all the gating tox jobs so that should be easy enough to do from d-g20:03
*** jcoufal has quit IRC20:03
clarkbmtreinish: perfect20:03
clarkbI need to grab food now, back in a bit20:04
mtreinishbut was there an improvement by increasing the concurrency?20:04
*** changbl has quit IRC20:04
SpamapShttp://logs.openstack.org/66/51966/7/check/check-tempest-devstack-vm-full/b61b229/console.html ... weird failure20:05
*** julim has quit IRC20:05
SpamapSoh never mind20:06
SpamapS2013-10-22 19:48:20.663 | Build timed out (after 60 minutes). Marking the build as failed.20:06
mtreinishSpamapS: it shouldn't take over an hour for a full tempest run though20:07
fungiyeah, what you want to do is look for any large gaps between timestamps in the log to see if some step/test took way longer than it should20:07
*** SergeyLukjanov has joined #openstack-infra20:08
*** SergeyLukjanov has quit IRC20:09
mtreinishfungi: that could be tricky though because the tests are in parallel. So if one thread is stuck on a long test the other threads keep printing20:09
mtreinishso there wouldn't necessarily be a large gap in the log20:10
fungimmm, good point20:10
fungii forgot i can't use that trick any more20:11
*** xeyed4good has quit IRC20:11
fungiand since jenkins kills it early, we don't get the slowest tests list out of testr20:11
fungiteasing out what causes jobs to run too long could take some thought now20:12
clarkbmtreinish: it helped on rax nodes but not so much hp20:12
*** xeyed4good has joined #openstack-infra20:13
*** xeyed4good has left #openstack-infra20:13
mtreinishfungi: well it looks like it took ~25min before tempest even started, and on the rax nodes the tests can take ~40mins20:14
*** melwitt has quit IRC20:14
fungiyep, in which case something in the devstack setup is to blame, so maybe gaps in the setup logs will be revealing20:14
fungior early in the console log while devstack does its stuff20:15
mtreinishfungi: it actually looks like devstack ran fine there is a ~10min gap between 'Started by user anonymous' and 'Running devstack'20:17
*** beagles has quit IRC20:17
mtreinishI think devstack normally takes around 10-15min20:17
mtreinish(including the devstack-gate part)20:18
*** sarob has joined #openstack-infra20:18
*** wenlock has quit IRC20:19
*** changbl has joined #openstack-infra20:19
*** wenlock__ has quit IRC20:19
*** zul has joined #openstack-infra20:19
*** sandywalsh has quit IRC20:20
*** alcabrera has quit IRC20:20
*** dcramer_ has joined #openstack-infra20:27
yolandahi fungi,i have a question about sso plugin and slaves, have you managed to make them work with sso enabled?20:30
*** vipul-away is now known as vipul20:31
*** rakhmerov has joined #openstack-infra20:31
*** hashar has joined #openstack-infra20:31
openstackgerritMatthew Treinish proposed a change to openstack-infra/elastic-recheck: Fix irc message for multiple bugs found  https://review.openstack.org/5321020:32
*** wenlock has joined #openstack-infra20:33
fungiyolanda: you mean sso in the jenkins master and having jenkins slaves authenticate to the master using sso?20:34
yolandafungi, well, i had one problem creating slaves but i was able to manage it, using admin user and api token to create the node20:35
yolandabut problem that i have now is that to activate it, it uses the .jnlp url for the slave, and then redirects to SSO, which is not what i want20:36
*** rakhmerov has quit IRC20:36
fungiyolanda: instead we configure our jenkins masters to connect to slaves using the ssh20:37
fungier, the ssh method20:37
*** melwitt has joined #openstack-infra20:37
annegentlefungi: heh, sure enough, one patch waiting to merge in operations-guide, https://review.openstack.org/#/c/53103/ :)20:38
yolandafungi, but you are creating nodes manually or using jenkins api?20:38
annegentlefungi: so my game plan I'd like to run by you for the ops guide when you have a moment20:38
fungiyolanda: we set the launch method option to "launch slave agents on unix machines via ssh"20:38
openstackgerritJiri Tomasek proposed a change to openstack/requirements: Add django-bootstrap-form to requirements  https://review.openstack.org/5236120:38
annegentleI put sentences together really oddly on IRC20:38
fungiyolanda: we add slaves via the api, yes, but not from the slaves themselves (they don't self-add)20:39
fungiannegentle: fire away. i have a couple minutes still before i have to duck out20:39
fungiannegentle: and clarkb and others may want to chime in as well20:40
fungiannegentle: also see http://eavesdrop.openstack.org/meetings/infra/2013/infra.2013-10-22-19.01.log.html#l-12620:41
annegentlefungi: so I'm going to take propose_translation_update_manuals.sh and modify for "propose update from O'reilly github to our gerrit"20:41
fungimake sure that jives20:41
* annegentle reads20:41
*** yolanda has quit IRC20:43
annegentlejust to clarify, the docs people get accounts for atlas20:43
annegentlethe original authors of the ops guide have them for example20:43
*** gyee has joined #openstack-infra20:43
*** yolanda has joined #openstack-infra20:43
*** dcramer_ has quit IRC20:43
annegentlenice that dhellmann knows the pain we're trying to avoid :)20:44
annegentley'all do understand you don't get "real" books without xml somewheres? :)20:44
yolandafungi, ok, i'll give a try20:44
*** Guest56169 has quit IRC20:45
fungiannegentle: latex isn't xml ;)20:46
annegentlefungi: heh neither is asciidoc20:46
fungiyou don't get *real* books without a typesetting language somewhere20:46
annegentlefungi: but once you index a book the markers are all over the place20:46
annegentlefungi: er, index markers20:46
annegentlefungi: so indexes make "real" books I guess20:46
fungiokay, so there will be some visibility into atlas from the authors. that's cool20:47
*** che-arne has joined #openstack-infra20:47
fungidoesn't sound like it's mandatory to the process though20:47
fungirather a nice-to-have20:47
annegentlefungi: I think the o'reilly editors won't know everything and will need authors to do edits20:49
annegentlefungi: but I'm not entirely sure20:49
yolandasorry, fungi, the jenkins create_node api doesn't allow to specify ssh as execution method, how are you specifying that?20:50
fungiso a discussion forum for the authors to communicate with and lead the editors20:50
*** alcabrera has joined #openstack-infra20:50
fungiyolanda: i'd have to read through the code in nodepool to find out20:51
fungibut i'm about to run away for a bit20:51
*** changbl has quit IRC20:52
yolandai see that there is a params dict there, but it cannot be specified in the api call20:52
fungiannegentle: so anyway, yes the translation proposal script has a similar skeleton to what i expect we'd need. it checks for updates on a remote site, retrieves them and pushes them into gerrit for review20:52
sdaguefungi / mtreinish: http://logs.openstack.org/66/51966/7/check/check-tempest-devstack-vm-full/b61b229/console.html#_2013-10-22_18_51_10_359 is suspicious20:53
annegentlefungi: should I go ahead and try a patch? Do you mind heavy reviewing?20:53
annegentlefungi: in case I get stuck20:53
sdaguethat being said, I also had jobs fail out on rax today at the 60 min timeout20:53
fungiannegentle: i don't mind at all20:53
sdagueso rax maybe got extra slow?20:53
*** SergeyLukjanov has joined #openstack-infra20:53
*** mestery has quit IRC20:53
sdaguemtreinish: concerning the 8x concurrency, I'm actually less concerned about overall run time, and more interested in shaking out more races20:54
sdagueand high concurency does that20:54
*** julim has joined #openstack-infra20:54
fungiannegentle: sounds like we have until the summit to get it implemented and tested, if we want to use it on the initial iteration (but it's also something which can be easily mimicked by a human initially if needed)20:54
*** mestery has joined #openstack-infra20:54
annegentlefungi: yes, that sounds right20:55
dhellmannannegentle: :-)20:55
*** julim has quit IRC20:55
*** Guest56169 has joined #openstack-infra20:56
*** Guest56169 has quit IRC20:56
fungidhellmann: were you a mathematician at some point? all the mathematics journals expected submissions in tex20:56
mtreinishsdague: why don't we try running 2 tempests at a time then? I'm sure that will shake loose races.20:56
dhellmannfungi: I generated latex from sphinx because they wouldn't take the rst directly20:56
annegentledhellmann: hehehe20:57
sdaguemtreinish: I suppose, but that's harder than just adding an arg :)20:57
*** atiwari has quit IRC20:58
*** mestery has quit IRC20:58
dhellmannannegentle, fungi : to be fair, Pearson would have taken docbook if I had it in that format, too20:59
annegentledhellmann: yeah o'reilly seems happy with us, even if we're at v5 and they're 4.5, they're transforming on the fly20:59
*** sarob has quit IRC20:59
*** Bada has quit IRC20:59
dhellmannannegentle: xslt to the rescue?21:00
fungiokay, i have to flee for a little while and eat dinner with other humans, but will be back in a couple hours i home21:00
fungier, i hope21:00
*** gyee has quit IRC21:00
*** jcoufal has joined #openstack-infra21:01
clarkbI am back21:01
*** Guest56169 has joined #openstack-infra21:01
*** hashar has quit IRC21:02
*** mestery has joined #openstack-infra21:03
*** changbl has joined #openstack-infra21:03
*** gyee has joined #openstack-infra21:03
*** cody-somerville has quit IRC21:04
*** hashar has joined #openstack-infra21:04
*** hashar has quit IRC21:04
*** julim has joined #openstack-infra21:05
annegentledhellmann: yeah they're good at xslt :)21:06
*** dkliban_ has quit IRC21:06
annegentledhellmann: they have asciidoc to docbook scripts so if we ever want to mass-transform the openstack docs we could :)21:06
annegentledhellmann: er, docbook to asciidoc21:06
annegentledhellmann: in their github repo21:07
dhellmannannegentle: I haven't really looked at asciidoc, does it compare favorably to restructuredtext?21:07
*** yamahata has joined #openstack-infra21:07
*** julim has quit IRC21:07
clarkblifeless: can you elaborate on how ci is involved with https://bugs.launchpad.net/openstack-ci/+bug/122470521:08
uvirtbotLaunchpad bug 1224705 in openstack-ci "pypi + --offline is still accessing tarballs on the internet" [Undecided,New]21:08
annegentledhellmann: supposed to be easy to write a bunch but then also gives you some structured "pointers" an example is http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/high-availability-guide/ap-cinder-api.txt21:08
lifelessclarkb: mordred tells me it is ci tooling limits that lead to requirements.txt files having urls rather than version references.21:09
lifelessclarkb: -and-21:09
clarkbwell its the bonghits oslo release process21:10
clarkbmordred: ^ I don't think CI is to blame for that, unless I am mising something21:10
lifelessclarkb: when we start doing CI of things we'll be enforcing offline builds work21:10
*** julim has joined #openstack-infra21:11
*** ericw has quit IRC21:11
dhellmannannegentle: I'll take a look, thanks21:14
*** sarob has joined #openstack-infra21:16
openstackgerritClark Boylan proposed a change to openstack-infra/config: Publis python-ironicclient tarballs.  https://review.openstack.org/5322221:18
*** zul has quit IRC21:21
clarkbsdague: dstufft Alex_Gaynor https://bugs.launchpad.net/openstack-ci/+bug/1232592 does that still need to be tracked?21:21
uvirtbotLaunchpad bug 1232592 in openstack-ci "pypi CDN bugs cause SSL errors when trying to install packages" [Undecided,New]21:21
*** weshay has quit IRC21:21
*** hashar has joined #openstack-infra21:21
Alex_Gaynorclarkb: As far as I know that's now resolved21:22
dstufftclarkb: that should be fixed now21:22
sdagueclarkb: yeh, I think we're good on it21:22
sdagueI'd say close it21:22
dstufftif you have any more problems let me know21:22
dstufftas that'd be a new problem :)21:22
*** yamahata has quit IRC21:23
*** dstufft has quit IRC21:25
*** dstufft has joined #openstack-infra21:25
*** melwitt has quit IRC21:29
*** rakhmerov has joined #openstack-infra21:31
*** mriedem has quit IRC21:32
*** dstufft has quit IRC21:32
*** dstufft has joined #openstack-infra21:33
*** dkranz has quit IRC21:34
*** rakhmerov has quit IRC21:36
clarkbsdague: did y ou have a response for https://bugs.launchpad.net/openstack-ci/+bug/120607321:36
uvirtbotLaunchpad bug 1206073 in openstack-ci "cinder volume fails to report in if jsonschema doesn't match what it was compiled with" [Undecided,New]21:36
sdagueclarkb: actually, that can get closed21:37
sdaguethat was the bug that made me go write the global requirements updater in devstack gate for 3 weeks21:37
*** vipul is now known as vipul-away21:37
sdaguei.e. solved by massively changing our infrastructure21:37
*** vipul-away is now known as vipul21:39
*** davidhadas has quit IRC21:40
clarkbsdague: and if you have time https://bugs.launchpad.net/openstack-ci/+bug/1218611 isn't that just a devstack/nova problem?21:40
*** ericw has joined #openstack-infra21:40
uvirtbotLaunchpad bug 1218611 in openstack-ci "Devstack uses a deprecated nova config option: LibvirtHybridOVSBridgeDriver" [Undecided,New]21:40
*** Guest56169 has quit IRC21:40
clarkbsince nova config comes from nova's repo and is edited by devstack21:40
*** ericw has quit IRC21:40
*** danger_fo is now known as danger_fo_away21:41
sdagueclarkb: actually, can you assign that to jog0, I think I just saw a patch for it21:48
sdagueand you can probably drop infra21:48
*** jpich has joined #openstack-infra21:49
*** julim has quit IRC21:49
*** julim has joined #openstack-infra21:50
*** ryanpetrello has quit IRC21:52
*** zul has joined #openstack-infra21:52
*** ericw has joined #openstack-infra21:53
*** dizquierdo has joined #openstack-infra21:55
*** miqui has quit IRC21:55
jerryzfungi: ping?21:58
clarkbpleia2: I am through the new bugs list. What do the numbers look like? I am going to swing back to medium and low bugs21:58
clarkbjerryz: fungi is AFK right now21:58
*** jcoufal has quit IRC21:59
*** hashar has quit IRC21:59
jerryzclarkb: i have a question about the topology of logstash21:59
pleia2clarkb: went from 190 to 175, so not bad :)21:59
clarkbjerryz: go for it21:59
*** melwitt has joined #openstack-infra22:00
*** jcoufal has joined #openstack-infra22:00
*** vipul is now known as vipul-away22:00
jerryzlogstash.o.o acts like an agent, e-r, and kibana22:00
jerryzis that the case22:00
jerryzwhile logstash-worker-n is indexer22:01
clarkbjerryz: ya22:01
jerryzclarkb: so it ships logs to indexer=>elasticsearch22:02
clarkblogstash.o.o is sort of the centralized endpoint for services built around logstash and elasticsearch. It also runs the gearman client and server that tell the log workers what to do22:02
clarkbpleia2: woot22:02
jerryzand as kibana and e-r retrieves results from elasticsearch22:02
clarkbjerryz: yup22:02
jerryzseems a loop to me22:02
clarkbjerryz: I don't understand22:02
jerryzwhy put the log source and log consumer on the same machine22:03
clarkbthe data flows in one direction from a service perspective, we just happen to have services at the front end of indexing and the backend on the same machine22:03
clarkbjerryz: why not?22:03
clarkb(in this case we have done that because the other machiens are resource hogs, but logstash.o.o isn't so it has become a home for related low cost services)22:03
*** hartsocks_ has quit IRC22:04
jerryzok. just curious about any benefits22:04
*** ruhe has joined #openstack-infra22:04
clarkbjerryz: you can compare http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=24 to http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=26 and http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=2922:05
clarkblogstash.o.o runs the lightweight services, the things behind it are not very lightweight22:05
*** ericw has quit IRC22:07
*** ruhe has quit IRC22:08
*** sarob has quit IRC22:09
*** Guest56169 has joined #openstack-infra22:10
*** harlowja_ has joined #openstack-infra22:10
*** harlowja has quit IRC22:11
*** vipul-away is now known as vipul22:12
*** melwitt has quit IRC22:12
*** blamar has quit IRC22:13
*** prad has quit IRC22:13
jerryzclarkb: logstash.o.o is a zmq client and gearman server at the same time, what if  there are too many jenkins logs at the same time, is this going to me memory hungry?22:14
clarkbjerryz: on that server it processing jobs per log file, so in practice the memory use isn't too bad22:14
clarkbjerryz: it can grow when things get busy but I haven't seen it run out of memory yet22:14
clarkbjerryz: the trick there really is we aren't keeping track of each log line, instead its just each log file per job22:15
clarkbjog0: mtreinish: can I assign https://bugs.launchpad.net/openstack-ci/+bug/1172387 to you? I think you guys have been doing that more than anyone else22:16
uvirtbotLaunchpad bug 1172387 in openstack-ci "document how to add logstash queries to bugs" [Medium,Triaged]22:16
openstackgerritSean Dague proposed a change to openstack-infra/config: enable enforcing of bash8 in the devstack gate  https://review.openstack.org/5323822:16
*** rcleere has quit IRC22:18
jerryzclarkb: is this no longer used? https://github.com/openstack-infra/config/blob/master/modules/openstack_project/templates/logstash/agent.conf.erb22:18
*** alcabrera has quit IRC22:18
*** melwitt has joined #openstack-infra22:19
clarkbjerryz: it isn't, cleaning up the cruft in the logstash puppet stuff is on my list of things to do22:20
*** dims has quit IRC22:20
*** Guest56169 has quit IRC22:22
*** sarob has joined #openstack-infra22:23
*** zul has quit IRC22:24
*** zul has joined #openstack-infra22:27
*** UtahDave has quit IRC22:27
*** ericw has joined #openstack-infra22:28
clarkbdhellmann: https://bugs.launchpad.net/openstack-ci/+bug/1178442 is the problem there that we need to install graphiz on the machines using the sphinx graphiz extension22:29
uvirtbotLaunchpad bug 1178442 in openstack-manuals "graph rendering problem in openstack doc" [Low,Confirmed]22:29
lifelesshow do channels get added to http://eavesdrop.openstack.org/irclogs/ ?22:30
clarkblifeless: they get added to the meetbot channel list in modules/openstack_project/manifests/eavesdrop.pp22:30
*** rakhmerov has joined #openstack-infra22:32
*** matty_dubs is now known as matty_dubs|gone22:33
*** zul has quit IRC22:33
openstackgerritlifeless proposed a change to openstack-infra/config: Add Neutron and TripleO channels to eavesdrop.  https://review.openstack.org/5324322:33
openstackgerritClark Boylan proposed a change to openstack-infra/config: Install graphiz on Jenkins slaves.  https://review.openstack.org/5324522:34
*** datsun180b has quit IRC22:34
*** zul has joined #openstack-infra22:34
*** harlowja_ has quit IRC22:35
*** harlowja has joined #openstack-infra22:35
clarkbI probably shouldn't fix bugs as I see them but I spend enough time grokking the bug that a fix is easy22:35
clarkblifeless: can you get markmcclain to +1 that change for neutron? we just want ot make sure we aren't stepping on toes22:35
*** derekh has quit IRC22:36
*** rakhmerov has quit IRC22:36
*** yaguang has quit IRC22:36
lifelessmarkmcclain: ^22:37
*** dims has joined #openstack-infra22:37
*** dizquierdo has quit IRC22:38
markmcclainlifeless: thank! that's been on my to do list for a while22:38
openstackgerritChris Krelle proposed a change to openstack-infra/config: Add Ironic to eavesdrop.  https://review.openstack.org/5324722:38
lifelessclarkb: there is your answer ;)22:39
*** ericw has quit IRC22:39
lifelessNobodyCam: your patch wil conflict with mine I think22:40
clarkbapparently when you have a bunch of projects on an lp bug it times out a bunch when you try updating one of those projects :(22:40
*** changbl has quit IRC22:40
lifelessNobodyCam: can you git review -d 53243 and then rebase your change on top of that?22:40
SergeyLukjanovlooks like now I have no enough permissions to create new branches in savanna repos and afaiu it's the common practice, isn't it?22:41
clarkbSergeyLukjanov: ttx and the release managers group has that access for openstack/* iirc22:42
clarkbreed: https://bugs.launchpad.net/openstack-ci/+bug/1182690 did you guys end up making that happen?22:43
uvirtbotLaunchpad bug 1182690 in openstack-ci "Add mailman templates in other languages" [Medium,Triaged]22:43
*** Loquacity has joined #openstack-infra22:43
openstackgerritChris Krelle proposed a change to openstack-infra/config: Add Ironic to eavesdrop.  https://review.openstack.org/5324722:44
NobodyCamlifeless: :) that should get it22:44
*** CaptTofu has joined #openstack-infra22:45
*** esker has quit IRC22:46
lifelessNobodyCam: thank you!22:47
*** blamar has joined #openstack-infra22:52
*** ericw has joined #openstack-infra22:53
*** senk has quit IRC22:53
*** mrodden has quit IRC22:54
harlowjaanyone one want to honorary +2 https://review.openstack.org/#/c/52462/ :) for me22:55
harlowjai give u guys the special honor of +2, lol22:55
sdagueso we seem to be hitting rax timeouts a lot today http://logs.openstack.org/23/53223/1/check/check-tempest-devstack-vm-postgres-full/6008b02/22:55
*** oubiwann_ has quit IRC22:56
sdagueclarkb: you have any thoughts other than us just bumping timeouts up to 90 minutes?22:57
sdagueor pulling the rax nodes out of the check queue22:57
*** harlowja_ has joined #openstack-infra22:57
clarkbI think those are our two options. I was skimming the timestamps in the console log and it appears to just be slow22:58
clarkbI don't see any crazy huge gaps indicating something isn't quite right22:58
sdaguethere is typically a big time sync at the beginning22:59
sdaguebut it's hard to figure out where it is going22:59
*** yaguang has joined #openstack-infra22:59
*** jpich has quit IRC22:59
sdagueby the time we get here - http://logs.openstack.org/23/53223/1/check/check-tempest-devstack-vm-postgres-full/6008b02/console.html#_2013-10-22_21_34_21_494 we've already burned 10 minutes22:59
*** harlowja has quit IRC22:59
*** thomasm has joined #openstack-infra23:00
clarkb sdague the first ~5 minutes we think may be related to jenkins problems23:00
*** thomasm is now known as Guest5589023:01
sdagueyeh, jenkins definitely feels pretty slow to me at this point23:01
clarkbsdague: its taking a while for things to get going, we saw similar with neutron python26 tests. theory is load related23:01
sdaguewhen I've tried to get to console logs in progress it's a long way23:01
*** zojoncj has quit IRC23:01
*** CaptTofu has quit IRC23:02
*** CaptTofu has joined #openstack-infra23:03
*** Guest55890 has quit IRC23:03
*** nicedice__ has joined #openstack-infra23:03
*** nicedice_ has quit IRC23:03
openstackgerritSean Dague proposed a change to openstack-infra/config: boost the devstack jobs to timeout at 90 minutes  https://review.openstack.org/5324923:05
clarkbsdague: yeah, I think when things get busy (particularly around the number of nodes) jenkins has a hard time with that. The boosted timeout is a decent workaround until we add more masters23:05
sdagueman jeblair needs to get working on making zuul + nodepool take over the rest of jenkins :)23:06
*** ericw has quit IRC23:06
*** wenlock has quit IRC23:08
*** pabelanger has quit IRC23:08
*** wenlock has joined #openstack-infra23:08
clarkblifeless: https://bugs.launchpad.net/openstack-ci/+bug/1220620 manage-projects does have some logging facilities now. It can't be configured to log to files, but can log at debug level to stderr or wherever the python logging default is. Is that sufficient to consider your bug fixed or would you like to be able to provide a logging config file23:09
uvirtbotLaunchpad bug 1220620 in openstack-ci "manage-projects is inscrutable on gerrit failures" [Low,Triaged]23:09
*** pabelanger has joined #openstack-infra23:09
clarkbpleia2: I have gotten through the bulk of the low and medium bugs, though I did a fair bit of skimming23:10
*** dcramer_ has joined #openstack-infra23:11
pleia2clarkb: thanks, nice work \o/23:12
clarkbpleia2: is there a final tally? I bet the number makes me feel sad23:12
pleia2so we fix released 17 of them23:14
lifelessclarkb: cool, thanks23:14
anteayaand I was next to no help on bugs today23:14
mgagneclarkb: FYI, this organization has no gravatar https://github.com/openstack-ci23:15
anteayaso it was really just pleia2, fungi and you clarkb23:15
anteayalooks like someone flipping the bird23:15
clarkblifeless: can I mark that fix released in your opinion? I want ot make sure that you feel it is fixed since you submitted it23:15
pleia2it has been one of my busier bug days23:15
*** reed has quit IRC23:15
anteayaworse than no gravatar23:15
clarkbmgagne: ok23:16
anteayapleia2: you were great23:16
*** reed has joined #openstack-infra23:16
anteayamgagne: there is one at openstack-infra: https://github.com/openstack-infra23:17
anteayawhen did the default gravatar become that image?23:17
*** julim has quit IRC23:17
pleia2would be nice to see openstack-ci go away23:17
pleia2anteaya: I'd totally see it as an berry with a stem or something, but now I can't stop seeing the bird, thanks ;P23:18
*** jcoufal has quit IRC23:18
anteayaberry with a stem, berry with a stem, berry with a stem23:19
clarkbpleia2: ++ to openstack-ci going away. Do you want to add that to next weeks meeting agenda?23:21
pleia2clarkb: sure, on it23:21
*** dkranz has joined #openstack-infra23:21
sdaguewhat's the bug tag again for fixes?23:22
clarkbsdague: I use 'Fixes-bug: XXXXXX'23:23
openstackgerritSean Dague proposed a change to openstack-infra/config: boost the devstack jobs to timeout at 90 minutes  https://review.openstack.org/5324923:23
clarkbI believe there are several variations but we seem to want to standardize on that one23:23
sdagueok, I'll amend that last commit with the bug pleia2 pointed out23:23
*** dcramer_ has quit IRC23:30
*** rakhmerov has joined #openstack-infra23:32
*** senk has joined #openstack-infra23:33
sdagueclarkb: so is there another +2er around for that patch?23:34
clarkbsdague: fungi I sort of around, I will just push it through (sorry didnt' consider I was the only one around)23:34
sdagueyep, no worries23:35
sdagueI just know folks are running into it :)23:35
sdagueas I've hit it 3 times today23:35
openstackgerritA change was merged to openstack-infra/config: boost the devstack jobs to timeout at 90 minutes  https://review.openstack.org/5324923:37
*** rakhmerov has quit IRC23:37
*** mriedem has joined #openstack-infra23:37
sdagueclarkb: awesome, now when I recheck my devstack patch for the 3rd time it will maybe pass :)23:38
sdaguegah, recheck doesn't work until the job fully times out23:40
*** Loquacity has quit IRC23:40
Ryan_Laneclarkb: have you had a chance to test the ES change?23:40
clarkbRyan_Lane: gah no, let me do that right now23:41
Ryan_Lanecool. thanks23:41
*** ericw has joined #openstack-infra23:41
*** Loquacity has joined #openstack-infra23:42
*** dcramer_ has joined #openstack-infra23:43
*** xeyed4good has joined #openstack-infra23:45
*** changbl has joined #openstack-infra23:46
clarkbRyan_Lane: it lgtm. There is a small diff in config files, but only 3 lines removed that are comments and blank so its safe. I will approve23:47
*** xeyed4good has left #openstack-infra23:47
*** cody-somerville has joined #openstack-infra23:47
*** sarob has quit IRC23:47
Ryan_Lanecool. thanks23:47
*** sarob has joined #openstack-infra23:47
Ryan_Lanewith this I think I can finally get the ES working for the wiki :D23:48
clarkbapproved, should get in for the 0000UTC puppet update23:50
*** vipul is now known as vipul-away23:51
openstackgerritA change was merged to openstack-infra/config: Configure elasticsearch via a hash  https://review.openstack.org/5304323:51
SergeyLukjanovI'm thinking about making a Savanna live demo for summit based on logs and it'll be great if some OpenStack-related logs will be used23:52
SergeyLukjanovit could be httpd logs or something like this23:52
clarkbSergeyLukjanov: there are all of the build logs on logs.openstack.org23:52
clarkbI think we have been wary of exposing other logs because we aren't sure how much sanitization they need23:53
*** matsuhashi has joined #openstack-infra23:53
SergeyLukjanovgood idea about logs.o.o, previously only thinking about httpd logs23:53
SergeyLukjanovlike access logs23:54
*** ericw has quit IRC23:54
clarkbdisclaimer: I spend a lot of time dealing with build logs and think they are generally the most useful logs we have23:54
clarkbs/useful/useful and interesting/23:54
SergeyLukjanovclarkb, yeah, agreed23:55
SergeyLukjanovare there any stats you want to know about it?23:55
clarkbnothing off the top of my head. I think we are still generally trying to sort out what we can do with elasticsearch. Hadoop was something I looked at too23:56
*** vipul-away is now known as vipul23:57
SergeyLukjanovclarkb, ok, thank you, will take a look at logs to understand what could be done with them :)23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!