Friday, 2013-09-06

*** svarnau_ has joined #openstack-infra00:01
openstackgerritA change was merged to openstack-infra/devstack-gate: Add support for large_ops tempest test
*** svarnau has quit IRC00:04
openstackgerritA change was merged to openstack-infra/config: Run the ceilometerclient tests under PyPy
openstackgerritA change was merged to openstack-infra/config: Removes separate Object Storage Admin manual as the content is now in other books.
*** thomasm has quit IRC00:05
mordredjeblair: any reason you didn't APRV this one: ?00:05
clarkbdon't ever give me a real chainsaw there won't be anything left00:06
mordredjeblair: (it looks like a great patch to me, but I wanted to make sure there wasn't a larger consideration)_00:06
openstackgerritA change was merged to openstack-infra/config: Add python 3.3 and pypy checks to ironicclient
mordredomg. wins the prize for using the word 'gerunds' in a commit message00:07
openstackgerritA change was merged to openstack-infra/config: Gerrit sysadmin tips for account repairs/renaming
*** Ryan_Lane has quit IRC00:11
*** Ryan_Lane has joined #openstack-infra00:11
openstackgerritA change was merged to openstack-infra/config: Enable translation jobs for the savanna project
*** pcrews has joined #openstack-infra00:23
jeblairmordred: it changes a job i don't have time to babysit00:27
mordredjeblair: nod00:29
*** Ryan_Lane has quit IRC00:32
*** dkliban has joined #openstack-infra00:34
*** UtahDave has quit IRC00:34
jeblairmordred: took 30 mins00:37
jeblairer 36 mins00:38
mordredjeblair: the current norm has dropped below that, hasn't it?00:38
jeblaircurrent runs on hp are taking 25 mins00:38
jeblairmordred: so about 1.4 times the hp time00:38
mordredwhich is roughly what the speed difference was before, wasn't it?00:39
jeblairmordred: i think we saw up to 2x, but yeah, i think about 1.5 was pretty common00:39
jeblairmordred: we should add dfw and ord in the same manner and benchmark them as well00:39
mordredjeblair: agree00:39
jeblairso we're not just going on memory00:39
clarkbmordred: jeblair: also with testr tempest ~36 minutes may be acceptable if we need that extra capacity00:46
mordredgood point00:46
*** reed_ is now known as reed00:46
*** reed has joined #openstack-infra00:46
*** svarnau has joined #openstack-infra00:46
mordred10 minutes is not nearly as bad a hit at 30 minutes when the test run 1 hours and 1.5 hours00:46
clarkbso I have really taken a chainsaw to the etherpad manifests00:47
clarkbThis is definitely not going to be backward compat00:48
*** svarnau_ has quit IRC00:48
*** darraghb has joined #openstack-infra00:48
clarkbbut I think it will amke what sdague has done easier00:48
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add test jobs for RAX DFW and ORD regions
mordredjeblair, clarkb: ^^00:48
darraghbquery on writing the param documentation for JJB: should current plugin defaults that are not explicitly set by JJB be noted or just skipped and marked as optional?00:49
* mordred not sure00:49
mordreddarraghb: how's australia?00:49
darraghbmordred: getting warmer, was 31C yesterday :) bit too warm to go running in the afternoons though00:50
mordreddarraghb: heh. yeah, I'll bet!00:50
darraghbquery above is in regards to:
mordredjeblair: ^^ opinion from you?00:52
darraghbNot sure whether to only specify as defaults what the JJB code specifies and mark others as optional, or note the current plugin defaults as well if not set by the yaml00:53
*** nosnos has joined #openstack-infra00:53
*** dina_belova has joined #openstack-infra00:54
darraghbmordred: rewritting the threading code sometime today since I spotted that there is now a delete-all option that could benefit from using similar code, should have sometime up today00:54
*** gyee has quit IRC00:54
*** rfolco has joined #openstack-infra00:57
*** dina_belova has quit IRC00:58
*** michchap has joined #openstack-infra00:59
*** ryanpetrello has joined #openstack-infra01:00
*** ryanpetrello has quit IRC01:07
*** changbl has joined #openstack-infra01:08
pabelangerjeblair, nodepool is licensed under apache 2.0?01:13
*** rfolco has quit IRC01:14
*** yongli_going_hom is now known as yongli01:19
*** ryanpetrello has joined #openstack-infra01:25
openstackgerritClark Boylan proposed a change to openstack-infra/config: Update etherpad and etherpad puppet manifests.
*** krtaylor has quit IRC01:28
*** dkehn has quit IRC01:30
*** marun has quit IRC01:33
*** dkehn has joined #openstack-infra01:35
*** svarnau has quit IRC01:37
*** ericw has joined #openstack-infra01:44
*** dims has quit IRC01:45
*** dina_belova has joined #openstack-infra01:54
*** lnxnut has joined #openstack-infra01:55
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Add repo scm
*** ryanpetrello has quit IRC01:56
*** sdake_ has quit IRC01:56
*** dina_belova has quit IRC01:58
*** sarob_ has joined #openstack-infra01:59
*** sarob has quit IRC01:59
*** ryanpetrello has joined #openstack-infra02:01
openstackgerritClark Boylan proposed a change to openstack-infra/config: Update etherpad and etherpad puppet manifests.
clarkbour apache rewrite rules are not working with new etherpad, other than that ^ seems to work02:03
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Add repo scm
clarkboh the database user stuff for root@::1 seems to break on hosts without ipv6 which is annoying02:04
*** ericw has quit IRC02:05
*** nati_ueno has quit IRC02:05
*** yaguang has joined #openstack-infra02:13
*** rcleere has joined #openstack-infra02:15
*** reed has quit IRC02:18
*** fifieldt has quit IRC02:19
*** sarob_ has quit IRC02:20
*** sarob has joined #openstack-infra02:20
openstackgerritKui Shi proposed a change to openstack/requirements: Unpin WebTest version in global-requirements.txt
*** ryanpetrello has quit IRC02:27
clarkbsuccess! and i have imports working but not exports02:27
openstackgerritClark Boylan proposed a change to openstack-infra/config: Update etherpad and etherpad puppet manifests.
*** xchu has joined #openstack-infra02:35
*** yaguang has quit IRC02:40
*** ericw has joined #openstack-infra02:42
clarkbmy test vm in hpcloud land can't talk to right now02:45
*** ericw has quit IRC02:45
clarkbat least not over http02:45
clarkbfor whatever reason my local test vm decided to stop trusting review.o.o02:53
*** pmoosh has joined #openstack-infra02:54
*** dina_belova has joined #openstack-infra02:55
clarkbsomething to be aware of if our d-g nodes in hp land start failing02:55
clarkbhmm works as root but not the ubuntu user, so confused02:58
*** mriedem has quit IRC02:59
*** yaguang has joined #openstack-infra02:59
*** dina_belova has quit IRC02:59
openstackgerritClark Boylan proposed a change to openstack-infra/config: Update etherpad and etherpad puppet manifests.
clarkbwell now I feel silly latest patchset fixes the cert issue03:09
*** jhesketh has joined #openstack-infra03:18
*** dims has joined #openstack-infra03:22
*** pcrews has quit IRC03:36
*** xchu has quit IRC03:47
*** rcleere has quit IRC03:54
*** dina_belova has joined #openstack-infra03:55
*** sandywalsh has quit IRC03:56
*** dina_belova has quit IRC04:00
*** vogxn has joined #openstack-infra04:00
*** nati_ueno has joined #openstack-infra04:01
*** pcrews has joined #openstack-infra04:05
*** lnxnut has quit IRC04:15
*** yaguang has quit IRC04:17
*** xchu has joined #openstack-infra04:18
*** yongli has quit IRC04:20
*** jhesketh_ has quit IRC04:24
*** xchu has quit IRC04:25
*** jhesketh_ has joined #openstack-infra04:25
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Use threads to update jobs in jenkins
*** yaguang has joined #openstack-infra04:34
*** sandywalsh has joined #openstack-infra04:39
*** xchu has joined #openstack-infra04:42
*** amotoki has joined #openstack-infra04:45
*** jergerber has joined #openstack-infra04:49
*** dina_belova has joined #openstack-infra04:56
*** nati_uen_ has joined #openstack-infra04:57
*** sarob has quit IRC04:57
*** sarob has joined #openstack-infra04:57
*** dina_belova has quit IRC04:58
*** dina_bel_ has joined #openstack-infra04:58
*** dina_bel_ has quit IRC04:58
*** yongli has joined #openstack-infra04:59
*** sarob has quit IRC05:02
*** pcrews has quit IRC05:09
*** sarob has joined #openstack-infra05:11
*** pmoosh has quit IRC05:15
*** sarob has quit IRC05:16
*** Ryan_Lane has joined #openstack-infra05:20
*** xchu has quit IRC05:31
*** nati_ueno has quit IRC05:33
*** nati_ueno has joined #openstack-infra05:34
*** nicedice_ has quit IRC05:36
*** xchu has joined #openstack-infra05:43
*** pblaho has joined #openstack-infra05:49
*** dina_belova has joined #openstack-infra05:59
*** dina_belova has quit IRC06:03
*** dina_belova has joined #openstack-infra06:09
*** dina_belova has quit IRC06:13
*** odyssey4me has joined #openstack-infra06:16
*** xchu has quit IRC06:20
*** nati_ueno_2 has joined #openstack-infra06:21
*** alexpilotti has quit IRC06:22
*** sarob has joined #openstack-infra06:22
*** nati_ueno has quit IRC06:25
*** sarob has quit IRC06:28
*** xchu has joined #openstack-infra06:33
*** tian has joined #openstack-infra06:55
*** dina_belova has joined #openstack-infra07:04
*** dina_belova has quit IRC07:09
*** vogxn has quit IRC07:13
*** alexpilotti has joined #openstack-infra07:13
*** vogxn has joined #openstack-infra07:14
*** zhiyan1 has joined #openstack-infra07:20
zhiyan1fungi: ping07:20
*** vogxn has quit IRC07:28
*** dina_belova has joined #openstack-infra07:31
*** jergerber has quit IRC07:35
*** mkerrin has quit IRC07:39
*** nati_uen_ has quit IRC07:39
*** dina_belova has quit IRC07:40
*** dina_belova has joined #openstack-infra07:42
*** mkerrin has joined #openstack-infra07:47
*** jpich has joined #openstack-infra07:49
*** boris-42 has joined #openstack-infra07:49
*** vogxn has joined #openstack-infra08:00
*** xchu has quit IRC08:03
*** dizquierdo has joined #openstack-infra08:10
*** xchu has joined #openstack-infra08:16
*** alexpilotti has quit IRC08:17
*** sld has joined #openstack-infra08:18
*** dina_belova has quit IRC08:24
*** alexpilotti has joined #openstack-infra08:26
*** derekh has joined #openstack-infra08:32
openstackgerritSteven Deaton proposed a change to openstack-infra/config: Added 'clean-remote' flag to jenkins jobs for docs site. Added 'clean-remote: true' to all ftp publisher jobs so as to clean out any stale / stagnant documentation files that may be left from prior OpenStack releases.
*** ruhe has joined #openstack-infra08:44
*** Ryan_Lane has quit IRC08:47
*** ruhe has quit IRC08:48
*** dkehn has quit IRC08:53
*** dkehn has joined #openstack-infra08:54
*** ruhe has joined #openstack-infra08:59
*** saschpe has quit IRC09:11
*** xchu has quit IRC09:12
*** dina_belova has joined #openstack-infra09:22
*** sandywalsh has quit IRC09:26
*** alexpilotti has quit IRC09:30
openstackgerritBoris Pavlovic proposed a change to openstack-infra/config: Add new project Rally
*** dina_belova has quit IRC09:35
*** CaptTofu has quit IRC09:38
*** CaptTofu has joined #openstack-infra09:39
*** vogxn has quit IRC09:42
*** alexpilotti has joined #openstack-infra09:42
*** CaptTofu has quit IRC09:44
*** yaguang has quit IRC09:44
*** vogxn has joined #openstack-infra09:52
*** alexpilotti_ has joined #openstack-infra09:57
*** dkehn_ has joined #openstack-infra09:57
*** alexpilotti has quit IRC10:00
*** alexpilotti_ is now known as alexpilotti10:00
*** pcm_ has joined #openstack-infra10:00
*** dkehn has quit IRC10:00
*** dina_belova has joined #openstack-infra10:03
*** echohead has quit IRC10:09
*** ruhe has quit IRC10:17
*** ruhe has joined #openstack-infra10:20
*** echohead has joined #openstack-infra10:24
*** darraghb has left #openstack-infra10:31
*** ruhe has quit IRC10:32
*** pcm_ has quit IRC10:32
*** pcm_ has joined #openstack-infra10:33
*** dkehn__ has joined #openstack-infra10:35
*** SergeyLukjanov has joined #openstack-infra10:37
*** vogxn has quit IRC10:37
*** dkehn_ has quit IRC10:39
*** sergmelikyan has joined #openstack-infra10:42
*** sergmelikyan has quit IRC10:53
*** vogxn has joined #openstack-infra11:00
*** dkehn__ has quit IRC11:03
*** dkehn__ has joined #openstack-infra11:04
*** zhiyan1 has quit IRC11:06
*** saschpe_ has joined #openstack-infra11:08
*** ruhe has joined #openstack-infra11:17
*** CaptTofu has joined #openstack-infra11:20
dizquierdohi fungi, I've got a doubt related to the review process, I guess you may help me (probably any core reviewer)11:23
dizquierdothe activity-board project is now under openstack-infra and I'm the only core reviewer there11:23
dizquierdobut also probably the only one committing changes there11:23
dizquierdoso, how should I proceed with changes? should I wait for a second reviewer opinion? (I guess I should...)11:23
sldimho, it's always good to get a second set of eyes on things.  (not that you asked. ;-) )11:24
dizquierdo just an example :)11:24
dizquierdoI guess so sld :)11:25
sldnot that my +1 matters much, but there you go.11:26
dizquierdohehe, thanks sld, just learning how this works, thanks ;)11:27
sldi sent something for review, myself, earlier this morning... the hard part is the waiting for a core reviewer to take a look at things. heh11:28
*** nosnos has quit IRC11:29
dizquierdoyep, my doubt was related to that, given that I'm the only core reviewer for that project :S (at least directly assigned, not sure if there are some meta-reviewers for the whole openstack-infra)11:29
sldi don't think it is project-related, per se, but moreso section and group related... for example - there are multiple projects in / under openstack-infra... and I think there is a group of core reviewers that look or can look at all projects in there.11:30
sldyeah....that's what i am thinking. (sorry, didn't read your whole message before I started typing mine. lol)11:30
*** vogxn has quit IRC11:38
*** vogxn has joined #openstack-infra11:44
*** dkranz has quit IRC11:51
*** vogxn has quit IRC11:56
*** adalbas has quit IRC12:01
*** anteaya has joined #openstack-infra12:04
*** ruhe has quit IRC12:07
*** ArxCruz has joined #openstack-infra12:08
*** weshay has joined #openstack-infra12:09
*** rfolco has joined #openstack-infra12:12
*** rfolco has quit IRC12:13
*** ruhe has joined #openstack-infra12:15
*** ruhe has quit IRC12:16
*** adalbas has joined #openstack-infra12:18
*** dkehn has joined #openstack-infra12:21
*** dkehn__ has quit IRC12:22
*** thomasm has joined #openstack-infra12:23
*** ruhe has joined #openstack-infra12:25
*** annegentle has quit IRC12:27
openstackgerritPeter Liljenberg proposed a change to openstack-infra/jenkins-job-builder: Added support for JaCoCo plugin Publisher
chmouelfriday fun
anteayachmouel: nice12:42
*** ruhe_ has joined #openstack-infra12:42
lifelesschmouel: LOL12:43
*** ruhe has quit IRC12:43
*** enikanorov-w has joined #openstack-infra12:44
lifelesschmouel: where do you find all these gifs?12:44
anteayachmouel is the gif master12:45
chmouelgoogle has filtering by animated images12:45
chmouelbut that one is thanks to
anteayaif I had to pick a gif I think of when I think of chmouel, this is the gif:
*** odyssey4me has quit IRC12:47
*** markmcclain has quit IRC12:47
lifelesswtf are you doing up anteaya ?12:49
lifelessI should go to bed I suspect'12:50
anteayait is 10 to 9am my time12:50
anteayatime to be at work12:50
lifeless0050 por moi12:50
anteayaah ha12:50
anteayawhich one of us should be asleep?12:50
anteayabut it is always nice to see you lifeless12:51
lifelessanteaya: ALL OF US12:51
anteayahey I'm for that12:51
anteayathe gate is empty12:52
lifelessit'ssat morming12:52
anteayanot yet for me12:52
lifelessI'm in the bar, not-exactly-sober working on talks for tomorrow, cursing mordred.12:52
anteayaha ha ha12:52
anteayasounds like the perfect kind of Saturday morning activity12:52
anteayain case anyone was wondering about guitarzan:
anteayalifeless: will your presentation be recorded?12:54
anteayanot only do I want to see it, I want to be able to point others to it12:54
lifelessanteaya: yes, it will be12:55
lifelessanteaya: formy sins12:55
anteayaI don't know that reference but... good?12:55
anteayafoamy sins12:55
anteayais that like drinking too much beer?12:56
lifeless-> because I'm a bad bad boy :)12:57
anteayaah okay, got it12:58
anteayawell you are a bad boy that churns out some code, so you have a few redeeming qualities12:58
lifelessanyhow, 1 beer is too much for me these days. Feed me spirits or cider or wine12:58
*** dprince has joined #openstack-infra12:58
lifelessspeaking of ;)12:58
* anteaya writes that down somewhere12:58
anteayayeah you did say you were in a bar12:59
lifelessanteaya: hotel one yeah; C and Lynne are asleep upstairs12:59
anteayaoh good, I am glad they are travelling with you on this one12:59
anteayawill they be in seattle too?12:59
lifelessbit far afield13:00
anteayaguess I am just going to have to come to NZ to meet them then13:00
lifelessyou should!13:00
anteayayeah, would be a long flight13:00
lifelessgreat skiing13:00
lifelessand stuff13:00
anteayanow we just have to come up with a plan that mordred will buy so I can go13:00
*** annegentle has joined #openstack-infra13:01
anteayalet's think of some group that will tolerate a presentation by me13:01
anteayawhat can I talk about and are they still accepting proposals?13:01
anteayai feel like I don't know anything, so it is tough for me to come up with a proposal for a talk13:02
lifelessI don't know and no.13:02
lifelessLCA miniconfs will be looking for talks13:02
anteayagreat, is there a url I can stare blankly at13:03
anteayaideas swimming in my head13:03
*** pcm__ has joined #openstack-infra13:03
anteayaor a url that leads to a page I can stare blankly at13:03
lifelessflamingspork is running a CICD miniconf13:03
lifelessmikal is running an OpenStack miniconfo13:03
lifelesscontinuous integration/C deployment.13:03
anteayawhat channel is flamingspork in?13:04
anteayaah yay13:04
*** pcm__ has quit IRC13:04
lifeless#drizzle often13:04
lifelessstewart smith13:05
lifelesssometimes known as 'stewart'13:05
anteayamikal - I would like to speak at LCA - the OpenStack miniconf - how do I apply?13:05
*** pcm__ has joined #openstack-infra13:05
lifelessok, I need to halt()13:05
anteayaokay thanks13:05
lifelessnight, or morning as the case() may be; all.13:05
anteayadon't see flamingspork there yet but I will lurk13:05
anteayahappy sleep lifeless13:06
anteayaand have a great presentation13:06
lifelesshe's probably still travelling from the burn.13:06
anteayaah okay13:06
*** pcm_ has quit IRC13:06
*** thomasbiege has joined #openstack-infra13:10
*** dina_belova has quit IRC13:11
sdaguejeblair: so I think I figured out the sparkline flicker thing. because you are wholesale replacing the html, I think you've basically lost all the img geometry, so the solution is going to be to change the way the rewrite goes so you are only replacing the inner html and then doing the src update on the images directly13:12
*** markmcclain has joined #openstack-infra13:13
*** ryanpetrello has joined #openstack-infra13:18
*** yaguang has joined #openstack-infra13:19
*** dizquierdo has quit IRC13:21
*** Bada has joined #openstack-infra13:22
*** derekh has quit IRC13:23
*** dims has quit IRC13:24
*** krtaylor has joined #openstack-infra13:29
*** derekh has joined #openstack-infra13:30
anteayaI found the page talking about miniconfs:
anteayathe only information I see about speaker selection is this line: The miniconfs are selected as part of the Call for Papers process which was run from June to July 2013. Which indicates to me that I am too late.13:34
boris-42somebody pls help=) already 2x 2+ (and already 2 times approved..)13:35
anteayahow interesting13:37
anteayait passed yet didn't merge13:38
*** dims has joined #openstack-infra13:38
anteayathe only thing i could do is post reverify and send it through the gate again13:38
anteayabut I am curious why it passed the gate yet didn't merge13:39
anteayamind if I wait for more eyes boris-42? they shouldn't be long13:39
openstackgerritA change was merged to openstack-infra/config: Add new project Rally
boris-42I wrote recheck no bug13:40
*** prad has joined #openstack-infra13:40
boris-42so it pass tests in that cases13:40
boris-42anteaya ^13:40
anteayarecheck puts it through the gate queue13:40
anteayayou want reverify13:40
boris-42I just periodically was typing git review, to check that it still could be merged13:40
anteayasorry I don't understand13:41
anteayaif you sent another comment to the patch while it was in the gate that could have affected how zuul dealt with it13:41
openstackgerritRyan Petrello proposed a change to openstack-infra/config: Add py26 and py33 tests and mirror uploads.
*** dina_belova has joined #openstack-infra13:43
*** eharney has joined #openstack-infra13:43
anteayaboris-42: I assume you want it merged?13:44
*** jhesketh has quit IRC13:44
boris-42anteaya it is already merged
boris-42anteaya but I wan't just to ensure that it could be merged (before pinging cores, so I was calling recheck no bug)13:45
anteayaI'm glad it is merged13:45
anteayaumm, well if you leave a comment with recheck on it, zuul puts it through the check queue13:46
anteayaso patch set 6 passed the check queue twice13:47
*** tstevenson has joined #openstack-infra13:48
*** ruhe_ has quit IRC13:49
*** vogxn has joined #openstack-infra13:55
*** burt has joined #openstack-infra13:56
*** jhesketh has joined #openstack-infra13:57
boris-42anteaya hmm there is no new project on stack forge is it ok?13:57
anteayawell the first thing that has to happen after your patch was merged, is that puppet needs to run13:58
anteayaand puppet runs jobs on a timer, not triggered by event changes13:59
anteayaso we have to wait for the timer to run first13:59
boris-42anteaya ah I should just wait=)13:59
anteayathen if it doesn't show up, we have to wait for a core to run manage_projects manually13:59
anteayayes, and keep an eye on it13:59
anteayaif you don't see anything after an hour, come back in and ask14:00
boris-42oh here is it!=)14:00
boris-42finally OpenStack CI=)14:00
anteayait looks very nice14:00
anteayacongratulations boris-4214:00
boris-42anteaya thanks14:01
*** pcm_ has joined #openstack-infra14:01
*** pblaho has quit IRC14:03
*** pcm__ has quit IRC14:04
chmouelsomebody knows which options of pip has 'install', '--pre',14:04
*** dina_belova has quit IRC14:10
*** dkranz has joined #openstack-infra14:11
mordredchmouel: uhm14:12
chmouelmordred: seems like 1.4.114:12
mordredchmouel: --pre doesn't exist until 1.414:12
chmouelyeah not 1.4 :)14:12
chmouelbut 1.4.114:12
mordredah, neat. I did not realize that14:12
mordredwell, then we all want to be on 1.4.1 at least then :)14:12
chmouelyep probably want to change devstack14:13
jeblairmordred: see conversation btwn lifeless and dachary in #jenkins; also
*** dina_belova has joined #openstack-infra14:15
mordredjeblair: neat14:17
mordredjeblair: are you thinking teuthology might be an interesting starting point for non-jenkins runners?14:19
*** pblaho has joined #openstack-infra14:19
jeblairmordred: i was indeed thinking that14:19
annegentleclarkb: jeblair: I still don't think Closes-Bug: nnnnnn is workign right for openstack-manuals.14:20
uvirtbotLaunchpad bug 1218375 in openstack-manuals "wrong network interfaced in docs" [High,Fix released]14:20
uvirtbotLaunchpad bug 1195571 in openstack-manuals "havana: nova - Editable default quota support" [Medium,Confirmed]14:20
annegentleThe commit messages have Closes-Bug: or Closes-bug:14:21
annegentlejeblair: and the email you sent said bug: nnnn should still work as if it is closes-bug14:21
jeblairdid i?  i thought it acted as Affects?14:22
jeblair(or Related or whatever it is... i haven't actually used it yet)14:22
sdaguehmmmm... any idea why this page isn't viewable - ?14:23
jeblairannegentle: i think we need to make hudson-openstack part of the group that is the bug supervisor for openstack-manuals14:23
jeblairannegentle: which means we need to make a group that is the bug supervisor, since it's just you right now14:23
annegentlejeblair: we can use the openstack-doc-core@lists.launchpad.net14:24
annegentlejeblair: (that was a question not a statement) :)14:24
jeblairannegentle: we usually have a -bugs group, why don't we set openstack-doc-bugs up?14:25
*** dina_belova has quit IRC14:26
*** thedodd has joined #openstack-infra14:28
annegentlejeblair: sure14:29
annegentlejeblair: can I add a group to that group?14:30
jeblairannegentle: ok, all set up now.  yes you can14:30
annegentlejeblair: ok done14:30
*** markmcclain has quit IRC14:31
jeblairannegentle: now the script should be able to assign bugs and change their statuses14:31
jeblairsdague: wfm14:32
jeblairsdague: you're comparing patchset 17 to base?14:33
chmouelmordred: how do I get tox to bootstrap pip with 1.4.114:33
chmouelit's always trying 1.3.114:33
mordredchmouel: you need new tox14:33
mordredchmouel: do you have tox 1.6 yet?14:33
chmoueli have 1.6.114:34
mordredhrm. and it's using pip 1.3?14:34
mordredthat's really weird14:34
mordredI'll have to poke for a minute14:34
*** pentameter has joined #openstack-infra14:34
chmouelyeah let me grep around if it's not my vm borked14:34
mordredchmouel: aiui, tox 1.6 should bundle/tie in the lastest virtualenv, which itself should bundle in latest setuptools and latest pip14:35
mordreddstufft: ^^ yeah?14:35
chmouelmordred: oh that was the part i forgot14:36
chmoueli had the one from the distro14:36
chmouelnot from whatever pip gives me14:36
mordredah. that's probably too old14:37
*** vogxn has quit IRC14:37
chmouel(it used to be more simple with  python tests/ :)14:37
mordredchmouel: fwiw, after this upgrade of pip/setuptools/virtualenv/tox, I don't forsee needing to do it again, so hopefully the fun we've had with this this cycle will not come back14:38
mordredchmouel: many things used to be more simple :)14:38
chmouelah nice14:38
*** alexpilotti has quit IRC14:38
*** weshay has quit IRC14:39
*** zeus has joined #openstack-infra14:40
*** ruhe has joined #openstack-infra14:41
fungimordred: jeblair: clarkb: i notice we're using openstack-ci-core and openstack-ci-admins somewhat interchangeably in gerrit acls and group inclusions. both have identical memberships at the moment, but i guess the thought is that core could be more inclusive later. should the acls using -admins currently get switched to use -core instead?14:43
*** vogxn has joined #openstack-infra14:44
mordredfungi: I believe some of that may be hysterical raisins14:44
mordredfungi: I believe one of them was people with launchpad super-admin ability and one of them was ci-core14:45
mordredfungi: but then when the lp auth change happened, I don't think we re-assessed14:45
fungiright, that sounds reasonable. i wonder if we should empty -core and add -admin as included by it, then switch all normal infra project acls to refer to -core instead of -admins14:46
jeblairfungi, mordred: i'd be in favor of changing the acls to use -core14:46
mordredjeblair: ++14:46
mordredjeblair: perhaps taking the oportunity to make a new group called openstack-infra-core and remove references to ci-core and ci-admins?14:47
fungimordred: perhaps14:47
jeblairsounds good to me14:47
mordredwow. I just tried to set a lp bug to fix released and got a timeout error14:48
fungii think i might also like to clean up acls that specify already-default values, and then possibly collapse identical acls to ease future management (we can always break them back out when one needs to be a special snowflake)14:48
*** markmcclain has joined #openstack-infra14:48
mordredfungi: ++14:48
* mordred supports fungi in this endeavor14:48
fungimordred: i get timeouts from lp click actions semi-often14:49
fungialso oops error pages14:49
*** _TheDodd_ has joined #openstack-infra14:49
mordredttx: is storyboard finished yet?14:49
ttxha ha ha14:49
fungimordred: particularly bad on bugs with too many bugtasks14:49
*** thedodd has quit IRC14:49
* mordred keeps trying to recruit people to work on it...14:49
anteayafungi so this might mean that as I gain competence in an area, I can get -core status for that project and improve one project at a time14:49
*** dina_belova has joined #openstack-infra14:49
*** sarob has joined #openstack-infra14:50
fungianteaya: well, those are projects with their own individual acls/groups in that case (e.g. jjb)14:50
*** thedodd has joined #openstack-infra14:50
*** thedodd has quit IRC14:50
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Run the marconiclient tests under PyPy
anteayaI'm trying to work on storyboard but I also keep getting recruited for other stuff - puppet-dashboard, refstack, owncloud...14:50
anteayafungi: ah okay, so no14:50
* anteaya tries to make progress on puppet-dashboard today14:51
anteayaoh and asterisk testing in 2 hours14:51
fungianteaya: no acls/groups are set in stone. if we need to add someone as a core on a particular project it's easy enough to individualize it14:51
jeblairso, given a list of images from python-novaclient, how can i determine if they are "my" images or "system" images?14:52
anteayaoh okay14:52
mordredAlex_Gaynor: you're just running like a beast on pypy on the client libs!14:52
anteayaI'll try to make -core on something easy before Christmas14:52
jeblairmordred: any ideas there? ^14:52
fungijeblair: i think it can show you which ones are private14:52
ttxmordred: I'll be back full speed on it as soon as I'm out of the tunnel. In 2 months and a half.14:52
mordredAlex_Gaynor: are you trying to get all of them pypy enabled by havana?14:52
anteayaI'm open for suggestions14:52
jeblairfungi: oh, any idea how?14:52
Alex_Gaynormordred: most of them "just work" out of the box, so my job is more mechanical than exciting14:52
fungijeblair: i'm looking. came up on a security vulnerability a while back where that wasn't being enforced properly ;)14:53
mordredjeblair: hrm14:53
*** _TheDodd_ has quit IRC14:53
jeblairfungi: that's the best way to learn about new software features.14:53
fungijeblair: surprisingly, yes14:53
*** thedodd has joined #openstack-infra14:53
jeblairheh, i hear graphite has clustering now.  :)14:53
ttxmordred: I would certainly welcome someone who actually knows Django and web UIs to work on it. My blatant ignorance of those things shall soon start to show.14:53
mordredjeblair: I'm not sure14:54
Alex_Gaynormordred: Possibly, I need to take inventory and see what I skipped so far, I couldn't figure out how to get the keystone tests going, and openstack-client needs PyCrypto which doesn't work14:54
mordredAlex_Gaynor: it needs PyCrypto? when did we start using that?14:55
Alex_Gaynormordred: :noclue:14:55
mordredAlex_Gaynor: when you say "I couldn't figure out how to get the keystone tests going" do you mean keystone or keystoneclient? and "tox" didn't work?14:55
Alex_Gaynormordred: keystoneclient14:56
fungiwas that pulled in to start providing a certificate authority trust bundle maybe14:56
*** thedodd has quit IRC14:56
mordredAlex_Gaynor: oy14:56
fungiin response to the whole not checking validity on https endpoint certs14:56
Alex_Gaynormordred: keystoneclient tests fail if I just run `tox` out of the box14:56
mordredAlex_Gaynor: that's quite odd - that's pretty much all we do in the gate14:56
Alex_Gaynormordred: let me flush the venv and try again14:57
Alex_Gaynorerr the .tox that is14:57
*** pabelanger has quit IRC14:57
fungijeblair: oh, nevermind, it was public vs non-public flavors, not images14:58
fungithough i wonder if nova image-show gives similar parameter details14:58
Alex_Gaynormordred: is there some background server I need running?14:58
*** thedodd has joined #openstack-infra14:59
*** sarob has quit IRC14:59
* ttx puts the music on and starts pushing H3 out14:59
mordredAlex_Gaynor: nope. unittests should be unittests14:59
*** sarob has joined #openstack-infra14:59
*** sarob has quit IRC15:00
mordredfungi, jeblair: nova image-show seems to have a metadata owner field15:00
*** sarob has joined #openstack-infra15:00
fungimordred: yes, i'm staring at the same right now ;)15:00
*** sarob has joined #openstack-infra15:01
fungiowner and owner_id, both of which seem to be the same15:01
jeblairmordred: i believe hpcloud has that but rackspace does not15:01
mordredand at least on hp cloud, metadata owner is None for system images15:01
mordredoh joy15:01
chmouelAlex_Gaynor: yeah on keystone the API test for v2 start a background server15:01
mordredwow. that's a win for interop15:01
jeblairmordred: it is also a number for some system images15:01
* anteaya pictures ttx dancing around his office15:02
anteayain a Hawaiian shirt15:02
Alex_Gaynorchmouel: I guess it doesn't do it correctly?15:02
mordredchmouel: this is keystoneclient15:02
fungiinterestingly, owner_id doesn't seem to be displayed for the stuff from hpcloud's global catalog (just owner)15:02
chmouelAlex_Gaynor: ah for keystonesclient i have never were able to run that on my mba15:02
Alex_Gaynorchmouel: encouraging! :)15:02
jeblairfungi, clarkb: rackspace has | metadata image_type                            | base                                 |15:03
mordredjaypipes: why did you guys think that vendor-defined fields for metadata was a good idea?15:03
mordredjaypipes: I blame you15:03
jeblairand | metadata image_type                            | snapshot                                                                                                                                                                                                                                                        |15:03
* chmouel is at the moment fighting with trying to run the keysotne tests even on a vm :-p15:03
mordredAlex_Gaynor, chmouel:
mordredthat's me running py27 keystone client tests after removing the .tox dir15:04
chmouelmordred: yeah you running linux you lucky15:04
mordredchmouel: yes. I run the operating system that we develop for15:04
jeblairmordred: any idea how i can find out what my "owner" id is in hpcloud?15:04
mordredjeblair: I'm guessing it's probably your account id, which is show in the management console15:04
mordredjeblair: it might also be in the json returned from keystone in the service catalog, but I'd have to look15:05
*** sarob has quit IRC15:05
mordrednow this is fun:15:05
mordredrunning=${PYTHON:-python} -m discover -t ./ ./tests --list15:05
mordredNo module named _bsddb15:05
mordredthat's in python 2.615:05
chmouelmordred: ah15:05
chmoueli got that on macosx15:05
chmouelfor a while15:05
chmoueland fixed this morning15:05
*** UtahDave has joined #openstack-infra15:05
mordredAlex_Gaynor: that's fantastic15:06
jaypipesmordred: I fought that tooth and nail.15:06
ttxjeblair: I'm getting certificates fail on https://jenkins*.openstack.org15:06
ttxare those self-signed ?15:06
mordredAlex_Gaynor: so, I'm going to blame a difference between osx and linux - probably in a dependent lib?15:06
fungittx: those are self-signed/snakeoil15:07
*** pblaho has quit IRC15:07
mordredjaypipes: well, then, I apologize that I did not help you fight more15:07
ttxfungi: do we plan to fix that ?15:07
mordredttx: no15:07
Alex_Gaynormordred: are these tests based on mocking http requests with some on disk representation of a response?15:07
jaypipesmordred: I pushed for a tag model...15:07
mordredjaypipes: how about a "this is the metadata you will return, and if you don't like it, tough" model?15:07
fungittx: since we're deprecating the intrface and trying to move users/tools away from interfacing with it, there wasn't a lot of reason to buy certs for those15:07
mordredttx: do you have a use case where you need it?15:08
mordredAlex_Gaynor: looking15:08
sdaguemordred: looking at logs in process15:08
ttxmordred: I'm using it to check when -tarball jobs complete15:08
Alex_Gaynormordred: I don't see any, but want to make sure, my first instinct is case-insensitive FS15:08
sdagueI end up having to go to jenkins a lot15:08
ttx(and that they succeeded)15:08
chmouelAlex_Gaynor: must be my lucky day, the keystoneclient tests are running for me on my mba
*** sarob has joined #openstack-infra15:09
*** sarob has quit IRC15:09
*** thomasbiege has quit IRC15:09
*** sarob has joined #openstack-infra15:09
mordredttx: but I mean is there something you're doing with that which needs a real cert chain?15:10
ttxmordred: not so much... making sure that my tarball information is not MiMed15:11
ttxso publishing cert fingerprints somewhere would work for me as well15:12
jeblairwell, the tarball itself isn't even behind https15:12
ttxjeblair: at some point we said we could store its MD5 in the job result, for my script to compare15:13
ttxbut I agree it's not the only weak area in the chain at this point15:13
jeblairah, then you'd have something to verify, certainly.  so....15:14
*** thomasbiege has joined #openstack-infra15:15
ttxjeblair: I should be able to go through the API once I make my script multimaster aware15:17
ttxso that I don't really need the UI15:18
*** dina_bel_ has joined #openstack-infra15:18
tstevensonjeblair: Could you take a look at review 44913 and let me know if I am on the right track?  I'll need to make adjustments to the config files in another repo for the bug to be closed.15:19
*** vogxn has quit IRC15:20
*** ericw has joined #openstack-infra15:21
jeblairtstevenson: i'd like clarkb to take a first look at that; i think he wrote that code and knows it the best, so i'd want him to weigh in on the approach.15:22
*** dina_belova has quit IRC15:22
*** rnirmal has joined #openstack-infra15:22
jeblairhe should be around here soon15:22
tstevensonjeblair: Thanks. Hoping to celebrate my first merge today. :)15:23
*** danger_fo_away is now known as danger_fo15:24
jeblairmordred, fungi, jaypipes: huh, hpcloud has an image_type field, but for snapshots only; so it looks like "metadata.get('image_type') == 'snapshot'" would work on both hpcloud and rackspace15:25
fungioh, cool15:25
*** thomasbiege has quit IRC15:25
jaypipesjeblair: on JayCloud, I use image_type = "snappy"15:26
pleia2phew, quite the scrollback for being out just one day15:26
pleia2good morning15:27
anteayamorning pleia215:27
anteayaUtahDave wanted to stop by and say hello15:28
anteayahe has booked next week to work with us15:29
UtahDaveHey, everyone!15:29
anteayato see what we can do to get salt really working this time15:29
jeblairhi UtahDave!15:29
anteayaI'll work with him and we will be asking many questions15:29
anteayahe also knows we have bug day next Tuesday and will do his best to help15:29
UtahDavehi, jeblair!15:29
anteayawe just wanted to give everyone a heads up15:30
anteayain case there are good logs for us to read15:30
anteayaregarding salt15:30
anteayaand good places for us to start15:30
jeblairwed and thurs are the best days for me next week; but hopefully others will be around on other days15:31
fungii'm here all week. don't forget to tip your waiters15:31
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add 'alien-list' command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add alien-image-list command
fungiUtahDave: though to brace you for what we're seeing, things seem to work fine initially and then after a few days salt starts chewing up all the ram on the master and most minions have hundreds of salt processes running on them and refuse to respond to salt commands any longer15:33
*** odyssey4me has joined #openstack-infra15:33
fungihopefully this is a familiar failure pattern15:34
UtahDavefungi: what version of Salt was running on that system?15:34
fungii believe we were installing the debs from salt rather than the ones from ubuntu, and constantly updating them15:34
*** pcrews has joined #openstack-infra15:34
UtahDavefungi: yeah, we did have an issue with that. zmq issue15:34
fungii know at least recently we updated them all again to deal with the key generation vulnerability (a month or two ago?)15:35
Alex_Gaynordhellmann: FYI: if you don't have mongodb installed and try to run teh ceilometer tests they don't bail when it tries to start mongo, they just have like 70000 failures of "Cannot connect to localhost:29000"15:35
UtahDavefungi: that's great.15:36
fungiUtahDave: so even with basically current salt releases and newly generated/enrolled keys it was still happening15:36
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support
UtahDavefungi: What OS was the host?15:36
fungiUtahDave: most of them (including the master) are ubuntu 12.04 lts (precise pangolin)15:37
fungithough some minions are centos 6.415:37
fungiall with current updates from their respective distros15:38
UtahDavefungi: Do you know if all the minion's keys were accepted on the master?  There was a memory leak issue where unaccepted minions attempting to authenticate with the master would cause zmq to not release ram for each connection15:39
*** odyssey4me2 has joined #openstack-infra15:39
*** odyssey4me has quit IRC15:40
mordredUtahDave: we actually pre-make the keys on the master and then copy them to the minion when we create the minion - so they _should_ all be pre-accepted15:40
*** tstevenson has quit IRC15:40
fungiUtahDave: yeah, they were all accepted. i manually accepted all of them when regenerating the keys, and every machine launched from that point on also auto-accepted the key on the master via the edits you made to the aunch script15:40
fungier, launch script15:40
mordredoh. he made edits to that already? cool15:41
UtahDaveright. I was just wondering if something went awry with that key generation process15:41
fungimordred: yep, that's his handywork there15:41
*** thedodd has quit IRC15:41
*** thedodd has joined #openstack-infra15:42
fungiUtahDave: i looked at it the other day when we were pulling it back offline for now, and there was nothing relevant pending15:42
*** tstevenson has joined #openstack-infra15:42
UtahDaveok, good to know15:43
fungiactually, looking back at the list, there was one machine with a key listed in /etc/salt/pki/master/minions_pre which is actually online and hadn't been accepted15:44
fungiso maybe that one was the cause?15:44
fungiwould just one unaccepted minion do it?15:44
UtahDaveYeah, if you were running a version of Salt that didn't have the fix, that minion would keep trying to auth. zmq would open a thread for each connection and never release it.15:46
*** dina_belova has joined #openstack-infra15:46
*** dina_bel_ has quit IRC15:47
*** swann has quit IRC15:47
*** mrodden has joined #openstack-infra15:47
sapersip testing soon?15:47
boris-42morder hi15:47
boris-42mordred hi one more time15:48
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add a hold command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add a delete command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add image-delete command
jeblairsaper: 1 hour 10 minutes from now15:48
*** markmcclain has quit IRC15:48
*** odyssey4me has joined #openstack-infra15:49
*** pabelanger has joined #openstack-infra15:49
*** dina_belova has quit IRC15:49
*** dina_belova has joined #openstack-infra15:49
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add alien-list command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add alien-image-list command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add a hold command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add a delete command
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add image-delete command
*** sarob has quit IRC15:51
*** odyssey4me2 has quit IRC15:51
jeblairnodepool is becoming useful!15:51
UtahDavefungi and mordred: I'm looking over the Salt installation manifests15:51
*** ruhe has quit IRC15:52
fungiUtahDave: cool. well we can pick it up next week when you have time of course15:52
fungiit's not something which needs to be solved asap15:52
UtahDavefungi: That sounds good.15:52
*** michchap has quit IRC15:53
boris-42mordred should I be already a Rally core?15:53
fungiUtahDave: and as always, we REALLY appreciate your help and input. thanks again!15:53
*** odyssey4me has quit IRC15:53
fungiboris-42: i'll check15:53
boris-42fungi thank you15:53
*** dina_belova has quit IRC15:54
fungiboris-42: looks like i need to re-run manage-projects. doing that now15:54
boris-42fungi btw how could I manage core team?15:54
fungiacl didn't get applied yet15:54
UtahDavefungi: so from what I understand, the next step was to set up the Jenkins server so that when the tests pass it can execute a Salt command that would cause the Salt Master to execute a puppet command on all the nodes.  Does that sound right?15:54
fungiboris-42: once the acl gets applied the group will automatically be created, empty but self-managed. then i add you as the initial member and you can add others through the gerrit webui (i'll give you the url for it when i add you)15:55
*** KennethWilke has joined #openstack-infra15:55
boris-42fungy seems that i made nasty thing15:55
fungiboris-42: oh?15:55
boris-42fungi I thought that I have to add rally-core team15:56
boris-42fungi or this is not connected with acl at all?)15:56
fungiboris-42: oh, no problem. it's just not used by gerrit, but you can use that as your bug triage team in lp or whatever you want15:56
mordredUtahDave: yes, that's the idea15:56
mordredUtahDave: or, rather, not the jenkinsmaster15:56
boris-42fungi aA ok =)15:57
mordredUtahDave: but the slaves - or something15:57
mordredUtahDave: not sure where the _best_ place to do it is - but something that is not the salt master needs to be able to tell the salt master to tell the minions to run a puppet command15:57
fungiUtahDave: what we discussed more specifically, is that we'll write a jenkins job which runs in zuul's post pipeline whenever a change merges to openstack-infra/config (or other infra projects of interest) and that tells the salt master to have all minions update their puppetry from our puppet master15:58
mordredyes. what fungi said15:58
UtahDavemordred: yeah, we can totally do that.  very easy to do.15:59
mordredUtahDave: woot15:59
mordredI like it when things are easy15:59
fungiand i guess also tells the puppet master to update its git clone of the config project too, before telling the minions to check in with it15:59
fungibut yeah, fairly few involved steps with known triggers15:59
mordredI'd say "hey, master, update yourself" followed by "hey, everybody else, update yourselves"15:59
*** Ryan_Lane has joined #openstack-infra16:00
jeblairmordred: can you update ?16:00
mordredjeblair: I can16:00
fungiwe mainly just need to know that the git pull has completed on the master before telling puppet agent to update on the other systems16:00
boris-42mordred fungi thanks =)16:00
UtahDaveOK, so I'll get a demo of that going here on a simple setup16:00
fungiboris-42: looks like you're the current only member of rally-core, and can manage it via,info16:01
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add test jobs for RAX DFW and ORD regions
UtahDavethen I'm sure I'll have a few questions to make sure it works on the config project16:01
boris-42fungi thanks for help )16:01
fungiboris-42: or,members is a little shorter16:02
jeblairmordred, fungi, clarkb:
jeblairmordred, fungi, clarkb: that change scares me a little bit; especially since in our experience, jeknins does very poorly when lots of things happen to it at once16:02
mordredjeblair: I believe they're using that in production internally, fwiw, but I haven't read the change yet16:02
*** mgagne has joined #openstack-infra16:03
fungithey is hp?16:03
jeblairmordred: are they doing it with the gearman-plugin with 1200 jobs and 150 slaves?16:04
mordredjeblair: perhaps making MAXTHREADS configurable?16:04
jeblairmordred: yeah, that's what i was thinking16:04
fungithat would be a good compromise16:04
mordredjeblair: I doubt they have gearman plugin and 150 slaves, I _do_ believe they have at least that many jobs16:04
UtahDaveso fungi mordred and jeblair: I'll have this ready for next week.16:04
*** nicedice_ has joined #openstack-infra16:04
fungisetting it to 1 would in theory preserve the current behavior16:04
mordredUtahDave: awesome! thakns!16:04
fungiUtahDave: awesome!16:04
*** thomasbiege has joined #openstack-infra16:05
mordredfungi: yeah- and it's possible that something like 5 or 10 might even be acceptable for us - and if it's configurable would be easy enough to test/tune16:05
UtahDaveI'll be lurking in here if any of you have any questions or anything16:05
jeblairmordred: (keep in mind a job change for us involves a moderately expensive function set calculation and re-registering the gearman functions, all of which happens inside of a lock)16:06
mordredjeblair: ah yes. good point.16:07
*** gyee has joined #openstack-infra16:07
ttxok, we are officially done with H316:08
ttxget wild :)16:08
mordredso I'd say configurable MAX_THREADS, and perhaps set the default value to 1 so that landing it doesn't bork us, and then folks who want to use multi-thread and just configure the value up16:08
* mordred runs around waving his arms in the air and foaming at the mouth16:08
ttxand celebrate. The amount of load we've seen in the last couple weeks was nothing short of amazing16:08
jeblairmordred: so to change one job means that g-p grabs a lock, iterates over 150 nodes, for each node, {iterate over 1200 jobs, and re-register}; so the idea of 100 jenkins threads all waiting on that lock could be a lot for us.16:08
*** odyssey4me has joined #openstack-infra16:08
jeblairttx: yaay!16:09
fungiit looks like passing it in [jenkins] maxThreads already overrides MAXTHREADS so maybe just setting MAXTHREADS=1 would be enough?16:09
mordredhats off to all of the people who were not at burningman16:09
fungiunless i'm misreading the _parallel_exec() method16:09
ttxjeblair: for some reason, bugfixing doesn't seem to create the same kind of load. Go wonder16:09
mordredfungi: oh, nice16:09
jeblairttx: it will when we have storyboard.  everyone is going to love fixing bugs!16:10
ttxjeblair: that's its main feature. It will make you love filing and fixing bugs.16:10
fungijeblair: i do actually believe bug mnagement will be much more pleasant when i no longer have to touch lp to do it16:10
ttxincluding flaky neutron gate tests.16:11
jeblairttx: actually, here's what i've learned -- if storyboard has a status page with lots of little progress bars and stuff, people will look at it all day long!16:11
ttxmake it shiny and they will come16:11
fungithat too. widgets keep them coming back for more16:11
*** SergeyLukjanov has quit IRC16:11
jeblairfungi: yes, i agree.  i may have been exagerrating, but i wasn't being sarcastic.  :)16:12
jeblairi definitely wasn't speeling.16:12
* ttx eows before collapsing in a coma16:12
jeblairttx: congrats!  have a nice coma!16:13
ttxI will!16:13
*** svarnau has joined #openstack-infra16:13
jeblairso, speaking of going crazy;  is up for review!  :)16:14
* anteaya sets up an IV for ttx16:14
fungiiv beer16:15
*** thomasbiege1 has joined #openstack-infra16:15
fungijeblair: and passing tests!16:15
*** thomasbiege has quit IRC16:16
*** thomasbiege1 has quit IRC16:16
zulactually ttx really loves cheese so if you give him alot of cheese then he will be soooooooo happpy16:19
*** dolphm has joined #openstack-infra16:21
*** pcrews has quit IRC16:21
*** jpich has quit IRC16:24
dolphmhow should developers be installing tox? it's not in openstack/requirements or any test-requirements.txt (yet?)16:24
dimsdolphm, i think i found the instruction here -
clarkbjeblair: looking16:28
dolphmdims: hmm, i didn't consider that it doesn't need to be inside the environment being tested16:29
clarkbjeblair: perhaps we can default to one thread for safety but give folks the opporunity to use more?16:31
clarkbtstevenson: looking at your change16:32
clarkbtstevenson: any chance you spun the bot up against a gerrit and irc server and watched it run?16:34
* anteaya adds ttx's love for cheese to the same card that notes lifeless' penchant for spirits, cider and wine16:34
clarkba weekend long coma sounds nice16:35
*** markmcclain has joined #openstack-infra16:35
*** changbl has quit IRC16:37
SpamapSyou know what would be a killer feature for git review? git review --tox  ... which would run tox on every patch in the series I'm submitting.16:40
SpamapSor just --run=tox16:40
*** CaptTofu has quit IRC16:42
*** CaptTofu has joined #openstack-infra16:42
*** Bada has quit IRC16:43
fungii'm not positive that belongs in git-review, but should be a pretty easy stand-alone script to write (for each patch not present in origin, repeat this command and stop on any nonzero return code)16:45
tstevensonclarkb: No, unfortunately I do not have access to the gerrit event stream for That's the only thing stopping me.16:45
fungitstevenson: in fact, you do!16:45
fungitstevenson: anyone can use their gerrit account to watch the event stream, just need to ssh to its api port with your registered ssh key16:46
*** CaptTofu has quit IRC16:47
fungisame socket, protocol and authentication which is used to submit patches for review16:47
tstevensonfungi: tried that. told me shell was disabled.16:47
fungiso if you feed the bot your gerrit account name and your ssh key, it should be able to connect and see events16:47
clarkbtstevenson: port 29418 :)16:47
clarkbtstevenson: overall though I like it, still working through the change, but if tested I could see this merging16:48
tstevensonclarkb: I followed the instructions from
jeblairtstevenson: or maybe you didn't give it the stream events command16:48
jeblairtstevenson: ssh -p 29418 gerrit stream-events16:48
tstevensonjeblair: That may be the case. Will try that.16:49
fungitstevenson: yeah, if you can submit patches you should be able to see the event stream. it's allowed for all registered users, no special permissions required16:50
openstackgerritElizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add git.o.o to list of ssl certificates we monitor
pleia2so while looking into this patch, I realized isn't https, which is not optimal with list admins dropping their passwords in, do we have plans here? shall I submit a bug report?16:52
mordredjeblair, fungi, clarkb: if you get bored,
*** dkehn has quit IRC16:53
fungipleia2: it probably ought to be https, but also keep in mind that listadmins use shared passwords and individual subscriber passwords are sent monthly in plaintext e-mail so... poorly-kept secrets at best16:54
pleia2fungi: fair enough16:54
sdaguepleia2: mailman is so terrible on security anyway, passwords are plain text stored, https probably gives people a false sense of security16:54
*** dkehn has joined #openstack-infra16:54
sdagueso basically what fungi said :)16:54
jeblairfungi: (i hope we disabled the monthly password reminders)16:54
fungipleia2: though i still personally support switching it to https with a cert16:54
fungijeblair: oh, good point, we very well may have16:54
funginow that i think about it, openstack lists may be the only ones i don't get spammed by on mailman day16:55
*** gyee has quit IRC16:55
jeblairspeaking of spam...16:55
jeblairclarkb: your email provider is rejecting cronspam for your account.16:56
clarkbtstevenson: for the dynamic method dispatching was there a reason for that? I am slightly worried about it because the data it "switches" on comes from an external source16:56
anteaya4 minutes to asterisk16:56
clarkbjeblair: woo, more incentive to use a non gmail MX16:56
clarkbjeblair: I have a fastmail account configured to accept email under my domain, should I go ahead and switch to that now to cut down on your cronspam? (assuming of course that fastmail isn't going to do the same thing)16:57
pleia2ok, filed a bug for when someone gets bored ;)
uvirtbotLaunchpad bug 1221842 in openstack-ci "Add ssl cert to" [Undecided,New]16:57
clarkbthis is my halfway step to running a real MX16:57
jeblairclarkb: i don't see why not.16:57
jeblairrussellb: ping16:59
tstevensonclarkb: Just thinking if there were any adjustment to types of events handled by gerritbot that there wouldn't be a need to keep adding if statements to handle them. Only an additional method to support the type.16:59
jeblairpabelanger: ping16:59
russellbjeblair: hi!16:59
openstackgerritKhai Do proposed a change to openstack-infra/config: link git sha in commit comments to the gerrit review
russellblet's play with phones!16:59
jeblairthere is the list of servers i've set up16:59
pabelangerjeblair: pong16:59
jeblairso that's 2, 8, 30 gb of ram each in dfw, ord, and iad rackspace datacenters17:00
jeblairand then 2,8,30 in hpcloud az117:00
*** dina_belova has joined #openstack-infra17:00
jeblairi think that's 2, 4, and 8 vcpus respectively17:01
russellbjeblair: where shall we start?17:01
jeblairi've just restarted asterisk on pbx.o.o (dfw-2)17:02
anteayaI am currently the only person in this conference17:02
*** CaptTofu has joined #openstack-infra17:02
jeblairso how about we dial into that for a baseline17:02
jeblairand then we can try the other vm sizes in dfw17:02
russellbanteaya: is the DID going to work on each?17:02
russellberr, jeblair17:02
jeblairrussellb: i'm hoping so, if i only run asterisk on one at a time17:03
russellbyeah, it should then17:03
jeblairrussellb: (and when we switch data centers, i expect i'll need to tweak something in the gui)17:03
russellbi went ahead and called via SIP ...17:03
clarkbjeblair: that is so awesome17:03
clarkbjesusaurus: ^17:03
jeblairclarkb, fungi, mordred: want to dial in?17:04
jeblairanyone else:
clarkbyes doing so now17:04
svarnaui'm in17:04
*** dina_belova has quit IRC17:05
* saper joined 600017:05
saperit's only 8000 narrowband?17:06
russellbsaper: it supports G.72217:06
* pleia2 dials in too17:06
saperrussellb: does not seem to be offered...17:07
clarkbtstevenson: I think I lean towards not doing dynamic dispatch because it makes me less paranoid17:07
russellbsaper: worked for me?17:07
*** nati_ueno has joined #openstack-infra17:08
saperrussellb: what I get17:08
saperrussellb: checking my SIP client and my FreeSWITCH now17:09
tstevensonclarkb: Sounds good. I was able to get access to the stream so I will adjust and test. Thanks!17:09
clarkbtstevenson: great, thank you17:09
*** ruhe has joined #openstack-infra17:09
russellbsaper: i called directly via a SIP client and it used G.72217:09
russellbjeblair: say when17:09
svarnau6000 again?17:09
pabelangersaper: pb a full sip trace please17:10
jeblairis up now17:10
*** pcrews has joined #openstack-infra17:10
jeblairand i'm dialed in17:10
saperrussellb: how many kHz?17:10
russellbsaper: for what, G.722?  16kHz17:10
saperrussellb: thx, wasn't enabled on my Android client17:11
anteayacan anyone hear me?17:11
anteayai can hear you yes17:11
anteayaI will dial in again17:12
saperrussellb: now got G722 8kHz17:13
pabelangerthat is choppy17:14
pabelangeraudio gains to high17:14
SpamapShrm, why is this review gone?
SpamapS references it17:14
uvirtbotLaunchpad bug 1214602 in heat "stack_list loads all resource from the database via the ORM" [Medium,In progress]17:14
anteayaI think my sound problem is related to this laptop17:14
clarkbSpamapS: possibly a draft17:15
*** gyee has joined #openstack-infra17:15
anteayait doesn't pick up usb sound cards or headsets, so not related to asterisk17:15
anteayathe laptop speakers and mike are what I am using now, so you can hear me using those17:15
clarkbSpamapS: yes status = d in the database17:17
jeblairshutting down dfw-817:17
SpamapSclarkb: ah, so it was converted to a draft later (hence the bug ref) ?17:17
clarkbSpamapS: no, I think the gerrit hooks see the drafts, but most other things do not17:18
* russellb in17:18
jeblairdfw-30 is up17:18
russellbpabelanger: hmm, i think your silence detection idea is a good one ...17:19
russellbpabelanger: after we try out some other servers to see if it makes any difference, i agree that should be the next variable to tweak17:19
clarkbSpamapS: checking for draftiness is something the hook scripts should probably do17:19
pabelangerrussellb: Ya, been dealing with it locally actually17:19
russellboh yeah?17:19
pabelangerand causing choppy audio, because the bandwidth spikes up and down (for me)17:20
russellbpabelanger: bandwidth should be constant in this case17:20
* saper 's in17:20
russellbpabelanger: just an optimization in conference mixing17:20
SpamapSclarkb: I think its fine to have it as a draft, it would just be nice if gerrit said "This exists but you can't see it." not "its gone OR you can't see it"17:20
sapercan you see connected?17:21
jeblairSpamapS: i would like it if we could just disable drafts because they never work like people expect17:21
SpamapSjeblair: agreed actually, I stopped trying to use them.17:21
SpamapSI'd actually rather have a --wip than a -D17:21
SpamapSas in, just submit it but immediately mark as WIP17:22
*** derekh has quit IRC17:22
pabelangersaper: yes, but your audio is very poor17:22
*** reed has joined #openstack-infra17:22
clarkbwe had that but had to revert it17:22
clarkbcould probably add it back in17:22
saperpabelanger: not saying anything17:22
pabelangersaper: talk now17:22
jeblairclarkb: except that we might be about to lose wip17:22
russellbi muted saper heh17:22
jeblairfrom gerrit17:22
russellbjust trying to kill echo17:22
clarkbjeblair: true17:22
russellbyou can unmute using *117:22
russellbin theory17:23
saperI'm on mobile phone using ear+mic17:23
russellbsaper: ah ok17:23
saperon wifi17:23
fungisaper: maybe input gain on the mic?17:23
pabelangerpress 717:24
russellbhold mic further away perhaps, heh17:24
fungiahh, actual phone17:24
pabelangerthat decreases talk volume17:24
russellbnow what17:26
pabelangerhave a meeting :)17:26
russellbcya pabelanger17:26
*** dina_belova has joined #openstack-infra17:26
russellbsorry saper muted you again17:26
saperrussellb: ok17:29
saperrussellb: too much bg noise?17:30
saperrussellb: will try from a narrowband cisco phone, at least wired, not wireless17:31
anteayasaper: and a lot of background noise17:31
*** ruhe has quit IRC17:32
anteayaoh sorry missed the bg in your question17:32
*** mkerrin has quit IRC17:32
saperwhat's the current IP address?17:33
russellbabout to switch17:33
jeblairwe're going to iad-2  next, but it'll take me a minute since we're switching data centers17:34
sapermaybe my kids did too much noise :)17:34
anteayaI have a lower pitched crystal bowl handy now, let me know if you want me to test with it17:35
*** dkehn has quit IRC17:36
russellbjeblair: what makes switching DCs take longer?17:36
*** dkehn has joined #openstack-infra17:36
jeblairrussellb: so has a a preferred pop for the did; i assume incoming calls would be routed through that pop to our server17:37
jeblairrussellb: or do you think they would just use whatever host it registered with?17:37
fungisaper: it seemed there was a combination of background hum/static, echo and the gain was so high that your voice was being overdriven17:37
russellbi guess i don't know for sure, but it should switch as soon as the new host registers17:37
* russellb blames freeswitch17:37
jeblairrussellb: well, just in case, let's wait 60 seconds (which is their propogation time)...17:38
saperfungi: interesting17:39
jeblairokay starting asterisk now17:39
saperconnected to 6000@
saperthe only person in this conference17:40
fungiseems to be up17:40
*** hemnafk is now known as hemna17:40
*** dkehn has quit IRC17:41
saperI think that might me wifi17:42
*** mdenny has joined #openstack-infra17:42
*** ruhe has joined #openstack-infra17:44
*** nati_ueno_2 has quit IRC17:45
*** nati_ueno_2 has joined #openstack-infra17:45
* saper has a dedicated, almost empty box if we want to test17:45
*** dkehn has joined #openstack-infra17:47
russellbpbx*CLI> channel originate SIP/6000@ application Milliwatt17:48
pabelangerand back17:48
russellbpabelanger: ohai17:48
*** UtahDave has quit IRC17:49
jeblairnext up: az1-2
*** nati_ueno_2 has quit IRC17:50
fungisaper: might make for an interesting baseline, but i think the bigger issue is proving we can do this with donated cloud virtual resources and have something which is still usable under load17:50
fungiso while a dedicated machine might help prove asterisk or our selected settings are sane, it doesn't help us much beyond that17:50
clarkbwe need something like openstack on moonshot providing 1:1 resources17:50
clarkbmordred: ^ you should get on that :)17:51
saperfungi: sure, I think I just setup "", no pin17:51
jeblairstarting asterisk now17:51
saperFreeSWITCH plays nice spanish music if you are alone in the conference :)17:52
jeblairsaper: are you going to connect?
*** vogxn has joined #openstack-infra17:55
saperjeblair: yep, just configued "az1" extension on my box17:56
saper2013-09-06 17:57:24.995889 [DEBUG] sofia.c:6282 Channel sofia/external/6000@ entering state [calling][0]17:57
russellbsaper: it's not going to work17:58
jeblairsaper: oops, we forgot to fix security groups, so the port is blocked17:58
saperrussellb: why?17:58
russellbsecurity group settings17:58
*** nicedice_ has quit IRC17:58
*** yaguang has quit IRC17:58
jeblairsaper: clark is going to try to fix it17:58
jeblairsaper: try now18:01
*** mrmartin has joined #openstack-infra18:01
saperI keep being disconnected18:03
saperafter ~30 secs or so18:04
pabelangersession timers?18:04
russellbyeah, some sort of SIP timeout18:05
russellbsaper: you can try TCP18:05
sapertrying again18:05
russellbthat'd probably avoid whatever the issue is18:05
pabelangerrussellb: tt-monkeys do it!18:05
* saper has bass sounds in the background, but I'm muted18:05
russellbpabelanger: on it18:06
saperchecking the disconnect reason18:07
russellbdid the conference die?18:07
pleia2I don't hear anything anymore18:07
jeblairseems like it18:07
jeblairi can't ssh into the host18:07
clarkbsame here18:07
russellbserver stopped responding18:07
*** dina_belova has quit IRC18:07
russellbmonkeys did it18:07
anteayamy connection is still open18:07
anteayathose monkeys18:07
saper2013-09-06 18:05:20.625874 [INFO] switch_rtp.c:3579 Auto Changing port from
saper0:13536 to are you NATTing?18:07
anteayanow my connection is closed, skype closed it18:07
clarkbsaper: yes hpcloud is NATing that server18:07
russellbit's the cloud!18:08
*** gerritTestMan has joined #openstack-infra18:08
*** gerritTestMan has left #openstack-infra18:08
clarkbI am still in the web ui for that account should I reboot the server?18:08
saper2013-09-06 18:05:52.324960 [DEBUG] switch_ivr_bridge.c:501 sofia/external/6000@15.185.18:08
saper127.85 ending bridge by request from write function18:08
saperthat's interesting18:08
jeblairclarkb: yes please18:08
anteayathat was the best sounding connection of the bunch18:08
anteayaso far18:08
saperlet me tcpdump it18:08
jeblairthe last thing i did before the server died was "tcpdump not port 2218:09
jeblairi'm not accepting responsibility for killing it though.  :)18:09
saperICMP host unreachable AH :)18:09
fungiyeah, whole server turned itself upside down apparently18:10
clarkbshould be up now18:10
clarkband I have the ocnsole log window open18:10
jeblairrussellb, pabelanger: do we need to set any asterisk params to deal with the NAT?  it is just straight NAT (not PNAT or anything)18:10
pabelangerI think russellb did something for it already18:11
russellbthat's all we should need to do18:11
clarkbthere is nothing in the log from when it went sideways18:11
russellbi think18:11
fungihpcloud doesn't give us a non-translated global address bound directly to the vif i guess18:11
saperthe only person @600018:12
saperat az118:12
jeblairi'm not having a good time using the shell18:12
jeblairi got the last login motd, but i don't have a shell yet18:12
clarkbjeblair: console log doesn't show anything18:13
fungidns resolvers misconfigured?18:13
clarkb*anything out of the ordinary18:13
fungioh, motd, so not rdns lookup timing out18:13
*** changbl has joined #openstack-infra18:13
jeblairfungi: well just the "Last login: Fri Sep  6 17:50:35 2013 from" line18:14
fungimaybe local disk i/o starvation trying to update wtmp/utmp18:14
jeblairfungi: nothing past that18:14
clarkbjeblair: I am having similar ssh problems18:14
jeblairfungi: it keeps updating the last login time, so i don't think that's it18:14
* saper is in18:16
jeblairi have a prompt now18:16
saper30seconds I'm gone18:16
jeblairand now when i log in again, i get one instantly18:16
jeblairthat was weird18:16
jeblair./sip.conf:nat = no18:17
jeblairrussellb: so i'll flip that, yeah?18:17
russellbthought i turned it on18:17
russellbcheck the node specific config18:17
clarkbpuppet may have changed it if puppet is running18:17
russellbyes that18:17
saperafter 30 secs or so I stop getting UDP packets18:17
russellbthat's equivalent to nat=yes18:17
russellbnat=yes is old school, heh18:17
jeblairrussellb: ok18:17
saperafter that I get18:18
saper18:17:14.542590 IP > SIP, length: 43018:18
saper18:17:14.548735 IP > SIP, length: 47918:18
jeblairsaper: try again, i have a tcpdump running on az1-218:19
fungii wonder if hpcloud's translation is simply useless for long-running datagram streams18:19
sapergone 18:20:04.09747218:20
clarkbfungi: could be, a lot of NAT systems try to treat them as stateful and derp all over them18:20
saperNAT timeout?18:20
mordredclarkb: already working on openstack on moonshot18:20
fungiit's one theory anyway18:20
russellbbtw fwiw, all of our calls involve a datagram stream18:20
saperI have no NAT on my SIP box18:20
russellbeach call has a separate datagraph stream of audio18:20
* mordred on way to airport - sorry not more help18:20
*** adalbas has quit IRC18:20
anteayamordred: happy trails18:21
clarkbrussellb: I understood18:21
fungimordred: i need openstack-moonshot-in-a-box for my basement for... um... purely academic purposes18:21
*** vishy is now known as vishious18:22
jeblairsaper: that was the end of your previous call18:22
fungiokay, so not necessarily the nat killing the sockets in that case18:22
*** thomasbiege has joined #openstack-infra18:22
pabelangerjeblair: *CLI> sip set debug18:22
pabelangersip set debug on18:22
russellbsip set debug ip <whatever>18:22
russellbso you don't get the whole world18:22
pabelangerthat too18:22
saperjeblair: ah, so you keep getting my udp packets?18:22
*** jog0 is now known as flashgordon18:23
jeblair18:20:01.124858 IP > pbx-az1-2.novalocal: ICMP udp port 28873 unreachable, length 3618:23
jeblairoh look at that one ^18:23
saperjeblair: getting those in the middle of the call18:23
*** adalbas has joined #openstack-infra18:23
saperI hear blurp or something but that ICMP does not disconnect18:24
jeblairsaper: can you call in now18:24
saperwhy it suddenly wants my media port plus one?18:24
jeblairsaper: i'm doing a full packet capture; let me know when you disconnect too18:24
pabelangerasterisk not getting a response back from 200OK18:25
russellbRetransmitting #7 (NAT) to
russellbit's not getting a response18:25
russellbso asterisk hangs up on you18:25
saperis it an attempt to do some NAT avoidance using another UDP port?18:25
russellbit's all like "yo, i answered your call, ACK ME BRO"18:25
russellbit's funny though, but it does respond to the  BYE18:26
fungiahh, so it's a control channel timeout causing it to drop the data channel18:26
russellbfungi: yep18:26
russellbhe sends INVITE, Asterisk responds with 200 OK (answers the call) ... and retransmits it a bunch of times, meanwhiel the audio stream is up18:27
fungireminds me of ftp ;)18:27
russellbit eventually gives up, sends BYE, and then receives a 200 OK in response18:27
russellbheh, SIP is a bit of a mess18:27
russellband that's being nice about it18:27
*** thomasbiege has quit IRC18:27
saperbut why 30 sec?18:28
fungiyeah, i used to support a rather large sip-based shoretel deployment as well as the data circuits and ipsec vpns interconnecting it. troubleshooting was always fun18:28
russellbfungi: ooh, sounds fun :)18:29
fungisaper: 30 seconds is presumably how long asterisk waits for a reply on the control channel before deciding your client is dead18:29
pabelangerfungi: saper: correct, timers18:29
saperbut we have control channel and media18:29
saperI can hear you guys and maybe you can hear me even18:30
pabelangersaper: you haven't ack'd the call control18:30
pabelangerso, asterisk tears down the call18:30
fungisaper: yeah, the data stream is getting through fine until asterisk decides to drop your call18:30
russellbso back to bridge testing ...18:30
russellbwhat have we learned?  heh18:30
saperbut it was not an issue with other hosts we tried18:30
clarkbhpcloud handles PSTN connections really well18:31
clarkbbut not sip potentially18:31
jeblairwhich is weird because they come in over sip18:31
fungisaper: right, i wonder if hpcloud is filtering your response on the control channel for some reason18:31
* saper downloads freeswitch.log to post18:31
* russellb blames freeswitch18:32
pabelangerI can try to debug sip from my home connection later today18:32
pabelangerwill fire up a few clients to see what happens18:32
russellbdropped me as well,18:33
fungicomparing packet captures at both ends, would be curious to see if that particular control channel response is transmitted but never arrives, or whether the 200 ok never gets to saper in the first place18:33
saperfull debug dump of a call
saperfreeswitch is also using mod_sofia18:34
openstackgerritAnne Gentle proposed a change to openstack-infra/jeepyb: Adds <service>-api to map to openstack-api-site in Launchpad
pabelangerjeblair: clarkb does this host have IPv6?18:37
pabelanger[2013-09-06 18:36:37.726] WARNING[22748]: chan_sip.c:4174 retrans_pkt: Retransmission timeout reached on transmission e0dfc6139b19ba0faba2cec45b9e642e@0:0:0:0:0:0:0:0 for seqno 1 (Critical Response) -- See
pabelangerrussellb: that IP is wrong18:37
pabelangersaper: are you using IPv6?18:38
* russellb gives up for now18:38
clarkbpabelanger: it does not18:38
russellbwe have IPv6 enabled in our configs18:38
pabelangermaybe try disabling it18:38
saperyes, I have ipv618:39
russellbor just setting on rackspace dulles18:39
russellbrackspace IAD, even the small instance, seemed to work OK18:39
russellbfor now18:39
saperand my freeswitch is using it probably18:39
pabelangerwell, asterisk could be confusing the IPv618:39
fungilack of ipv6 is a big sticking point for me when it comes to offering public-facing services from hpcloud anyway18:40
clarkbthe host does have a link ipv6 address18:40
pabelangerif we don't plan on routing it, we can just disable it in asterisk18:40
clarkbbut not a global scope address18:40
clarkbfungi: I think rackspace is one of the few that offer ipv618:40
clarkbamazon doesn't even do it18:41
clarkbnot to the VM at least I think you can have IPv6 VIPs on their load balancers18:41
pabelangerso, test ipv6 disabled?18:41
clarkbpabelanger: sure, I have to run now though18:42
russellbyeah me too18:42
pabelangersaper: try your call again18:43
saperjeblair: seems like NAT to me?18:43
*** russellb is now known as rustlebee18:43
sapertalking does not help18:44
pabelangeris a routing issue18:44
pabelangerstill not ack'ing packet18:44
saperchecking previous conversations logs18:44
*** ruhe has quit IRC18:44
pabelangerokay, need to stop and start the conference to properly change the bindaddress18:45
pabelangerso, people need to drop for a sec18:45
jeblairpabelanger: yep18:45
pabelangerokay, back18:46
tstevensonclarkb: Having some issue getting my own instance of gerritbot running. When I go to run it and pass my small test config as an argument it just dies silently. Not seeing any activity in the logs as to why.18:46
pabelangersaper: want to try again18:47
tstevensonclarkb: I've added the log_config path to my main gerrit config. Also added that information to the documentation.18:49
pabelangerpbx-az1-2*CLI> [2013-09-06 18:35:05.680] NOTICE[22748][C-00000028]: chan_sip.c:25381 handle_request_invite: Call from '' ( to extension 'sip:6000' rejected because extension not found in context 'public'.18:51
saperI'm in from mobile18:52
pabelangersaper: drop the sip: from the extension18:52
*** thomasbiege has joined #openstack-infra18:52
*** cp16net has left #openstack-infra18:52
saper50.139.221.52 is not me18:52
jeblairsaper: so if you look in your tcpdump logs, do you get the 200 OK for your invite?18:53
*** sdake_ has joined #openstack-infra18:53
saperjeblair: let me get a full tcpdump saved, w818:53
fungi50.139.221.52 looks like someone in the charleston, sc region using comcast18:54
jeblairsaper: it should be: you send an invite request, then asterisk sends '100 trying' then '200 ok'18:54
*** thomasbiege has quit IRC18:55
saperchecking the dump18:56
saperah didn't get a whole call18:57
jeblairsaper: how's it going?19:01
*** vipul is now known as vipul-away19:05
jeblairsaper: let us know if you have a good tcpdump19:05
*** atiwari has joined #openstack-infra19:05
jeblairsaper: we want to switch to another server and try some settings there19:05
jeblairsaper: (and you can also call that server for comparison)19:05
pabelangersaper: can you call again please19:07
pabelangersaper: still there?19:10
*** tstevenson is now known as codenamekt19:13
*** sarob has joined #openstack-infra19:14
anteayachecking dependencies for puppet-dashboard, I can get ruby1.9.3 through apt-get (calling it ruby1.9.1) but when I load rubygems through apt-get is it going to pull in ruby1.8:
anteayawhat is that music/sound called?19:14
anteayadid eveyone else leave the call?19:15
* saper is back19:15
*** codenamekt is now known as tstevenson19:16
fungii assume it's a music generator?19:16
* saper hears the music19:16
saperand now in19:16
fungianteaya: i wonder if there's a separate gems deb for 1.9.x19:16
jeblairsaper: pabelanger is changing another setting19:17
jeblairsaper: stick around and we'll ask you to call back in a minute19:17
anteayafungi: if I can find it, can we use the deb?19:17
*** sarob has quit IRC19:17
saperI'm sending INVITE, getting 100 Trying19:17
saperand then many times 200 OK19:17
pabelangersaper: okay, try again19:17
*** sarob has joined #openstack-infra19:17
jeblairsaper: are you sending anything in response to the 200 ok?19:17
*** CaptTofu has quit IRC19:18
sapernot sure19:18
saperlooks strange19:18
saperbut RTP seems fine19:18
fungianteaya: looks like the rubygems1.9.1 package is just a virtual package name provided by the ruby1.9.1 package anyway19:19
jeblairsaper: try calling again; pabelanger changed another setting19:19
*** Vivek has quit IRC19:19
anteayafungi: I don't quite understand virtual packages yet19:19
saperlet me check19:19
anteayaI have ruby 1.9.1 package (ruby.1.9.3) installed, no rubygems19:19
sapermaybe I try to respond to
anteayaI'll see if I can find a deb for it19:19
saperthat's what I got in "Contact"19:20
pabelangerthat is what I am changing19:20
*** vogxn has quit IRC19:20
pabelangerokay will update puppet19:20
fungianteaya: says it includes the /usr/bin/gem1.9.1 executable19:20
anteayayou are correct it is there, it responds to gem list but not rubygem -v19:22
anteaya$ gem -v19:22
anteayagem it is called gem not rubygem, thanks fungi19:22
jeblairpabelanger: i'll let you make the change on pbx.o.o and start asterisk19:23
pabelangerjeblair: okay19:24
jeblairpabelanger: i've already updated to use the other pop19:24
pabelangerback up19:25
pabelangersaper: ya, the issue was with our configs and the az1 zone19:26
*** Vivek has joined #openstack-infra19:26
clarkbwhat was the problem?19:27
*** reed has quit IRC19:27
pabelangerclarkb: nat, we bind to public IP on rackspace, not on az119:27
*** Vivek is now known as Guest6395919:27
pabelangerso, asterisk needs updating to know the external IP19:27
jeblairmordred: hpcloud feedback: nat is evil19:27
pabelangerso, depending on which provider we use, puppet changes need to happen19:28
SpamapSis there a "the cloud was too slow" bug that we should do rechecks against?19:29
SpamapSSeeing a few where it just took 400s for the database to even respond to a request... guessing it was just resource starvation.19:30
clarkbin tempest?19:31
fungiSpamapS: we've talked about having something benchmark each devstack slave before deeming it available for use (and discarding unusably slow machines)19:32
clarkbwe are running tests in parallel now could also be self generated load19:32
jeblairis anyone else around to call into the conference server?19:33
clarkbI am not19:34
pleia2I can19:35
pleia2rolex, score19:35
*** pcrews has quit IRC19:36
*** reed has joined #openstack-infra19:41
fungipleia2: i think you you didn't press 3 quickly enough, so no rolex for you19:41
fungioh, rolex was 2 anyway19:43
*** jhesketh has quit IRC19:43
pabelangerthat is 2019:43
pabelangersounds like ducks19:44
anteayaducks selling viagra19:44
pabelangermight want to mute a phone jeblair19:45
jeblairall of my phones are muted now19:45
jeblairit's someone else i think19:45
pabelangerthat is the down side of silence suppression19:45
anteayaI'm muted19:45
*** jhesketh has joined #openstack-infra19:45
anteayawas that me?19:45
pabelangerthink so19:45
anteayalet's test19:46
pabelangerand back19:46
anteayaI'll mute again19:46
anteayais that me?19:46
pabelangeranteaya: I believe so19:46
anteayabecause I hear the noise19:46
fungii mistook anteaya for a white-noise generator ;)19:46
anteayaI don't usually when I generate it but it could be19:47
anteayafungi: oh I generate lots of that19:47
anteayaI'll stay muted19:47
anteayawhy would that just start now though?19:47
anteayaguess my microphone generates white-noise at a low level and when silencing is off it is picked up19:54
anteayathe sound card in this laptop could be better19:54
anteayanice music19:56
pabelangerokay, back up19:57
sapersrry it was my19:57
saper16kHz music it was19:57
anteayait was nice19:58
saperit's used here for holding19:58
anteayaI am in and unmuted19:58
anteayashould I mute?19:58
jeblairpabelanger, rustlebee: thanks for your help!20:07
pabelangerjeblair: np.  I'll do more testing tonight with hp cloud, and get some patches ready if you decide to go that route20:08
jeblairpabelanger: cool; is still configured to use the dallas did, which is far away from hpcloud, but that probably won't bother you too much.20:10
*** sarob_ has joined #openstack-infra20:11
*** mrmartin has quit IRC20:11
*** sarob has quit IRC20:15
openstackgerritAnne Gentle proposed a change to openstack-infra/config: Adds info about building a Jenkins slave with script
openstackgerritAnne Gentle proposed a change to openstack-infra/config: Flattens directories based on
clarkbsysadmin email addresses are in hiera now... took me long enough to figure that out20:23
clarkbjeblair: I am updating my email address now, cronspam should hopefully get to me20:23
jeblairclarkb: yeah, if we put them in puppet, we get spam from random peoples servers that use our exim module20:23
clarkbyeah I remember now20:23
* anteaya goes for a walk20:24
jeblairso it's "private information", which isn't really a stretch.20:24
*** pabelanger has quit IRC20:25
*** UtahDave has joined #openstack-infra20:25
clarkbjeblair: new address is in hiera20:25
openstackgerritAnne Gentle proposed a change to openstack-infra/config: Updates user guide and basic install to point to new pom.xml file
zulis there a way to watch new commits on github like subscribe to all the commits for a particular project20:28
fungienrique granados has magic fingers on those strings20:29
openstackgerritClark Boylan proposed a change to openstack-infra/config: Add backup restore docs.
fungioh, he was the composer20:29
*** eharney has quit IRC20:29
clarkbfungi: jeblair ^ how is that20:29
clarkbzul: if you have control of the project github can do callbacks on events to http servers for you20:30
*** ftcjeff has joined #openstack-infra20:30
annegentlejeblair: clarkb: looks like I'll need a core approval for before I can flatten the directory structure for the Ops-guide20:31
annegentlechicken-and-egg -- the flatter structure won't pass the gate until the gate is updated with the new pom.xml file location. oh joy20:31
annegentlejeblair: maybe we should do this next week when we sit next to each other?20:32
fungizul: add a script to cron.hourly to pull from that repo and echo the results of nonempty pulls20:32
fungislight delay, but also helps batch them up20:32
zulfungi ill just install an rss reader :)20:32
fungizul: ahh, they have an rss feed. problem solved20:32
annegentlejeblair: clarkb: fungi: perhaps if you approve we can test in a small batch (that was my hope originally, start small!)20:33
fungibaby steps. got it20:33
lifelessanteaya: LCA miniconf selection works like this: CFP selects conference papers and *what* miniconfs. Then the miniconf organisers run their own CFPs.20:34
fungiannegentle: so the thought is to merge 45489 and then recheck 45487 to see if it passes?20:34
annegentlefungi: yep20:35
lifelessanteaya: also some papers submitted for the main conf are forwarded to the miniconf organisers as being suitable fodder20:35
* annegentle loves baby steps20:35
lifelessanteaya: so yeah yuo need to talk to mikal and steward/flamingspork20:35
*** dprince has quit IRC20:38
*** dkliban has quit IRC20:43
*** KennethWilke has quit IRC20:51
*** pabelanger has joined #openstack-infra21:00
clarkbwho was just in the room21:04
clarkbI should've checked irc before he left oh well21:04
*** boris-42 has quit IRC21:05
*** rnirmal has quit IRC21:06
clarkbannegentle: reviewing21:07
*** ruhe has joined #openstack-infra21:07
clarkbjeblair: did you want to review that before I approve?21:08
*** pentameter has quit IRC21:12
*** vipul-away is now known as vipul21:14
*** sarob_ has quit IRC21:14
*** sarob has joined #openstack-infra21:15
jeblairannegentle: ping21:16
clarkbjeblair: annegentle's change has my +2, but I have held off on approving21:16
*** pcrews has joined #openstack-infra21:17
jeblairclarkb: i was wondering if it could be made slightly more generic for easier jjb templating21:18
clarkbjeblair: good idea21:18
jeblairso this is for a manual that's in its own repo21:18
clarkbif the dir structure is consistent the {name} could potentially determine everything else21:18
jeblairwhich is bound to be a little different from the manuals that are built from openstack-manuals21:19
*** prad has left #openstack-infra21:19
jeblairbut i think there's at least one other that's similar to the ops guide, right?21:19
clarkbjeblair: yes, the translated doc job iirc21:19
jeblairclarkb: what repo is that?21:19
clarkbI thought it was the ops repo21:20
jeblairoh i see21:20
jeblair(i wonder if it needs changeng even for this change)21:21
clarkbjeblair: I don't think it works yet, but will need changing21:21
clarkbactually maybe not21:21
openstackgerritKhai Do proposed a change to openstack-infra/config: update thirdy party testing how to doc
clarkbit appears to use the generated dir which is already less deep21:21
jeblairok i think the ops guide is the only separate repo atm21:22
jeblairso my suggestion is largely theoretical21:22
jeblairbut even so, if the root pom were either at "/pom.xml" or something generic like "/src/pom.xml", the same definition could be used for a subsequent repo21:24
clarkbthe doc jobs could use templatization though, but this may not be a good place to start21:24
*** yjiang5 is now known as bmqq12321:24
*** thomasm has quit IRC21:25
jeblairi'm going to approve but leave that suggestion for later consideration21:25
*** ruhe has quit IRC21:25
dkranzjeblair, clarkb : Got a minute to chat about all the bogus ERRORs in the logs and how to get rid of them?21:26
jeblairi mean, we could still template it, it's just a mostly useless parameter to add21:26
mordreddkranz: they should go away21:26
dkranzmordred: May a suggest a way?21:26
dkranzmordred: We were going to file bugs and create a whitelist21:27
mordreddkranz: I have no suggestions, but I support the effort21:27
mordreddkranz: when you say whitelist - what do you mean?21:27
clarkbdkranz: no, you need to talk to dhellman and jd__21:27
clarkbor do you mean in general?21:27
clarkbcontenxt switching on a friday afternoon is not going well for me21:28
* mordred throws salmon at clarkb21:28
openstackgerritA change was merged to openstack-infra/config: Flattens directories based on
dkranzmordred: Then hopefully you guys could put something in devstack-gate to fail the build if any new ones show up.21:28
dkranzmordred: hrm about which part :)21:28
*** danger_fo is now known as danger_fo_away21:28
dkranzclarkb: In general. It is not just the new ceilo thing!21:29
dkranzclarkb: There are 99 ERROR in n-cpu after a success21:29
fungi99 errors but failure ain't one21:29
clarkbright most (maybe all) ceilo traces21:29
dkranzclarkb: No21:29
dkranzclarkb: I wish that were the case. I just looked.21:30
*** NobodyCam_ has joined #openstack-infra21:30
mordredfungi: if you're having nova problems I feel bad for you son21:30
dkranzclarkb: You saw the link in #openstack-qa a moment ago?21:30
*** odyssey4me has quit IRC21:30
*** NobodyCam_ has quit IRC21:30
clarkbdkranz: I see it now21:30
jeblairdkranz: i've forgotten, are we working from the assumption that there should be no errors in logs?  ie, it's never legitimate for a test to cause one?21:31
dkranzclarkb: Some of that seems like simple bugs and others seem nasty21:31
dkranzjeblair: Well, they are blackbox tests21:31
*** dolphm has quit IRC21:31
fungii thought the objection was more to presence of exception tracebacks when the test itself is succeeding just fine21:32
dkranzjeblair: IMO, there should only be an error in the log if it indicates a problem with the system21:32
clarkbdkranz: that particular trace appears to be a result of the level we log at21:32
dkranzfungi: What do you think a developer means when does log.error?21:32
jeblairdkranz: and is tempest legitimately able to cause system problems such that there should be error logs?21:32
fungidkranz: good point21:32
clarkbdkranz: it is a debug trace21:32
* mordred thinks that it's never legitimate for a test to cause one21:32
mordredan error in the log should indicate an operational error in the software21:33
mordrednot an invalid request from a user21:33
dkranzmordred: exactly21:33
clarkbmordred: unless you are logging at debug level21:33
jeblairyes, we're all agreed on that, that is not what i'm asking21:33
clarkbI think it is perfectly valid to see a trace at debug level21:33
dkranzclarkb: I thought debug trace meant you get debug messages21:33
dkranzclarkb: Not error messages that would not have shown up at another level21:33
clarkbdkranz: that is what you have there its a debug message + the corresponding trace21:33
jeblairdkranz: i'm trying to think about the whitelist -- is it only for grandfathering current errors and should trend to zero, or do we need to have a way for someone writing a new tempest test to indicate what error messages it is expeceted to cause?21:33
dkranzjeblair: I think it is just until the bugs are fixed21:34
mordredjeblair: I read it as grandfathering and trending towards zero21:34
dkranzmordred: Yes21:34
jeblairok, so it doesn't much matter where it is, the system could be as simple as grep and a list of regexes in a script in the devstack-gate repo21:34
dkranzjeblair: I think so21:34
clarkb(trace having its own level is a oslo.logging thing and isn't correct imo)21:34
dkranzjeblair: That is actually what I was thinking21:34
fungiwe do similar things for the oom and sudo checks21:35
jeblairclarkb: it's alse extremely confusing -- trace, when it exists in logging systems, usually means "level that's even finer grained than debug"21:35
*** bmqq123 has quit IRC21:35
mordredyeah. like, I'd think trace would be "magically show me all function enters and exists in sequence with indentation to indicate stack depth"21:35
clarkbjeblair: it goes debug <- trace <- info <- warn <- error or somesuch21:36
mordredwhich would be EPICLY verbose, but somethings super the hell useful21:36
mordredclarkb: I consider that a bug in the design of oslo.logging21:36
clarkbmordred: right21:36
clarkbmordred: there shouldn't be a trace level at all imo21:36
flashgordonmordred: with 28MB of n-cpu logs from a tempest run you would thing we do ta ht now21:36
mordredsimilar to the config files override command line arguments :)21:36
mordredflashgordon: :)21:36
clarkbmordred: because, it isn't standard and means I have to special case openstack logs :)21:36
flashgordonI think we are getting off topic here no?21:37
dkranzmordred: I think the main thing is we want operators (and tempest) to have something to run on the logs that says "no problem"21:37
clarkbflashgordon: not entirely. I still assert that tracebacks like that may be valid21:37
*** adalbas has quit IRC21:37
clarkbflashgordon: because we are explicitly logging at that level21:37
mordreddkranz: yes21:37
flashgordonclarkb:to me stacktrace == sharting21:37
dkranzclarkb: Then people have to know that, and what the valid ones look like.21:37
clarkbflashgordon: its debug info21:38
mordredI'm ok with stacktraces in debug output - sometimes it's helpful debug info21:38
clarkbthe ERROR you see first is what normal users will see21:38
flashgordonclarkb: lets take a concrete example
fungiperhaps debug output shouldn't go to the test console and should instead be captured separately (additional debug log files or something)?21:38
jeblairerrors should have tracebacks, but there should be no errors, so i think we can ignore it21:38
flashgordonI see no value in that stacktrace21:39
clarkbflashgordon: that is the one I am looking at21:39
flashgordonor the glanceclient loggin for that matter21:39
jeblairflashgordon: there is no value in that stacktrace only because there is no value in the _error_21:39
clarkbflashgordon: in general TRACE level stacktraces are fine imo21:39
flashgordonjeblair: agreed21:39
*** sarob has quit IRC21:39
dkranzSorry to start a fight and run but I just noticed the time and I'm late to leave.21:39
clarkbbecause of what TRACE means21:39
jeblairdkranz: i think we all agree21:39
fungihappy fridaying, dkranz21:39
dkranzsdague and I are interested in helping with this21:39
clarkbdkranz: yes I agree (I just think we need to articulate what is allowed)21:40
jeblairdkranz: i think it's fine to stick something in d-g to accomplish this21:40
flashgordonjeblair: \o/21:41
dkranzclarkb: Yes, some instructions for developers need to end up in HACKING or somewhere.21:41
mordredjeblair: ++21:41
clarkbbecause it should be valid to log a stacktrace during a passing test at your most verbose log level21:41
dkranzfungi: Same to you. Later...21:41
clarkbbecause info like that is what debug log level exists for21:41
clarkbsimilar to how glance was logging all of its sql queries21:41
clarkb(and still can if you choose)21:42
flashgordonI would love to see this logic in place for havana21:42
jeblairflashgordon: icehouse?21:42
flashgordonjeblair: havana, any issue we find is a bug21:42
flashgordonunless there is FF on infra too21:42
jeblairflashgordon: nope, and it's likely to be a relatively simple system, so i'm sure we can get it in.  i'm not sure how well people would react to that kind of change at this stage though.21:43
jeblairflashgordon: nope -> nope there is no infra ff21:43
clarkbjeblair: flashgordon: ya I expect the biggest problem being that people will need to be convinced that there are bugs that have to be fixed21:43
mordredflashgordon: have I talked to you about moving HACKING.rst to doc/source/hacking.rst and having it publish to ?21:43
clarkbat least before havana releases21:43
mordredclarkb: we can convince them by filing the bugs and then putting in a gate :)21:44
flashgordonjeblair: I am a fan of public shaming of projects when all else fails21:44
mordred(honestly, I don't think anyone disagrees that it's a problem, do they?)21:44
flashgordonmordred: no  but that is a great idea21:44
mordredessentially, anyone who has ever tried to diagnose a devstack problem...21:44
mordredflashgordon: woot21:44
flashgordonand the whitelisting will make this easier to do at first21:45
flashgordonbaby steps21:45
*** dkranz has quit IRC21:45
anteayasaper listening, lovely thank you21:46
clarkbmordred: I still think we need to be careful about how we do it21:46
*** mgagne has quit IRC21:46
clarkbflashgordon's example is one that I don't think should cause a failure21:47
clarkbnormal logging levels log an ERROR which tests may cause then debug level spits out a traceback21:47
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Publish hacking docs to
mordredflashgordon: ^^21:47
mordredclarkb: I never think that tempest should ever cause ERRORs to be logged21:47
mordredin a successful frun21:48
clarkbmordred: aren't their negative tests?21:48
mordreddoesn't matter21:48
flashgordonI agree with mordred21:48
mordredthere are negative tests for behavior21:48
mordredthat test that  requested invalid image does not return or something21:48
clarkbhow else do you capture that something went wrong in your logs?21:48
mordredthose are not errors in the software21:48
mordreddifferent things21:48
clarkbit may be that we fail if ERROR level happens and allow stacktraces below that21:48
mordreda user error and a software error are different21:48
clarkbthat makes sense to me21:49
mordredI think that eventually any ERROR logged by the software should be an indication that something failed21:49
mordredthat way, an operator knows to go dig into a problem if an error is logged21:49
fungiapache has ruined people into thinking that errors in what the client requests should be treated as errors in the server, and mixes them accordingly21:49
fungiat least where its logs are concerned21:50
anteayalifeless: thanks, just skimmed the backscroll21:50
fungithough you thankfully have error code ranges to fall back on to tell the difference in most cases21:50
anteayaI appreciate the introduction and will followup with mikal and flamingspork21:50
mordredanteaya: flamingspork's IRC nick is actuall stewart21:51
anteayayeah just met him in #drizzle21:51
anteayabut I really like typing flamingspork21:51
anteayajust like I like typing guitarzan21:52
anteayasome nicks just need to be typed21:52
anteayamy other favourite is niftylettuce21:52
flashgordonclarkb:  I see it as unexpected errors vs expected errors21:52
anteayadon't know him but that is a great nick21:52
*** mancdaz has left #openstack-infra21:52
*** mancdaz has joined #openstack-infra21:52
clarkbflashgordon: but do you log them differently? mordred's assertion is that you do21:54
clarkband I am on board with that21:54
flashgordonclarkb: works for me21:54
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: Poke a hole through the proxy for cluster health
clarkbflashgordon: the problem was originaly framed as stacktraces shouldn't end up in our logs and I think that is wrong. I am on board with no ERROR level logs should end up in the logs during tempest testing21:56
flashgordonwhich would mean only stacktraces that are explicitly raised are ok?21:57
flashgordonclarkb: I can get behind that21:57
flashgordonI am just not sure what case we have for that21:57
clarkbflashgordon: its similar to glance and the sql logs21:57
flashgordonclarkb: can you fill me in on that one21:58
clarkbflashgordon: until recently glance logged sqlalchemy at the debug level when its log level was set to debug21:58
clarkbflashgordon: so you got everything out of sqlalchemy21:58
mordredjeblair: are you going to submit the openstack ci talk for pycon?21:58
clarkbflashgordon: they are now individually toggleable so that glance=debug does not imply sqlalchemy=debug21:59
clarkbflashgordon: but the functionality wasn't completely removed because sometimes you need to log running state21:59
flashgordonclarkb: ahh yes21:59
mordredclarkb: btw - you never review my patches anymore :( I have patches in the queue going back to aug 10 than have gotten no love.22:00
* mordred pouts22:00
clarkbmordred: I haven't reviewed much stuff over the last few days22:01
clarkbso everyone is suffering22:01
mordredclarkb: what, you've been BUSY???22:01
clarkbmordred: did you see the chainsawed etherpad manifests?22:01
mordredclarkb: saw they went up - haven't looked yet22:01
clarkbmordred: I am actually pretty happy with how it turned out. I am considering possibly using a PPA for nodejs instead of building from source22:02
clarkb not sure what we need to do to vet that though22:02
clarkbanyone know chris-lea?22:02
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Move gate-tempest-devstack-vm-large-ops to silent
*** pcm_ has quit IRC22:03
jeblairmordred: yes, i'll submit the openstack ci talk for pycon22:04
mordredjeblair: great. I'll do one on pbr/packaging then22:04
flashgordonwhich pycon is this?22:04
mordredflashgordon: US - next april22:04
*** burt has quit IRC22:04
flashgordonoh montreal22:04
clarkbflashgordon: are you going to openstack on ales?22:05
flashgordonclarkb: yup22:05
*** ericw has quit IRC22:05
fungichris lea was apparently vp of business development at media temple. i don't know him, but sounds like a suit ;)22:05
flashgordongiving the triple talk22:05
ryanpetrelloanybody feel comfortable w/ approval for ?22:05
clarkbflashgordon: cool, I will give the CI talk. I am planning on driving there which may or may not be a mistake. But drives in and around the cascades are always fun22:06
flashgordonnot sure if I got the patch above right ^22:06
flashgordonclarkb: nice  where you driving in from?22:06
mordredryanpetrello: it terrifys all of us22:06
flashgordonI am flying in from bay area22:06
mordredryanpetrello: your javascript exceeds me22:06
clarkbflashgordon: seattle22:06
flashgordonahh niceshould be a gerat drive22:07
ryanpetrello@mordred oh come on, a little javascript never heard anybody22:07
*** w_ has joined #openstack-infra22:07
ryanpetrellowow, it's so late in the day my brain is making serious leaps here22:07
flashgordonclarkb: I gave this talk at europycon
*** zeus has quit IRC22:08
flashgordonwhich is CI from the developers side22:08
anteayamordred "flashgordon: have I talked to you about moving HACKING.rst to doc/source/hacking.rst and having it publish to ?" you talked to me about that and getting sphinx to generate docs for that repo22:09
anteayaand I haven't done it yet22:10
clarkbflashgordon: is the one I will give, need to update it though22:10
*** olaph has quit IRC22:10
flashgordonclarkb: nice22:10
*** olaph has joined #openstack-infra22:10
clarkbflashgordon: the developer focus is neat22:11
flashgordonclarkb: it was fun going back in time looking at our tests and how we did it in the beginning22:11
anteayamordred upvoting 45498 is easier than waiting for me to do it22:12
*** w_ has quit IRC22:13
clarkbit has come a long way :)22:13
*** w_ has joined #openstack-infra22:14
*** olaph has quit IRC22:15
*** lcestari has quit IRC22:16
clarkbI am going to walk home in hopes that a break will help me tackle code review later22:18
openstackgerritRyan Petrello proposed a change to openstack-dev/pbr: Add a hint for users who don't have git installed.
annegentleclarkb: jeblair: fungi: this spreadsheet has the changes to the directories spelled out, I do like the idea of templating but mostly right now just need to not break the gate before boot camp :)22:19
fungiannegentle: makes perfect sense22:19
*** tstevenson has quit IRC22:21
*** thedodd has quit IRC22:22
annegentlefungi: I hope so :)22:23
annegentlefungi: flattening is good22:23
annegentlefungi: I hope it doesn't throw translators off22:23
fungisince those are showing up as git renames, hopefully transifex is smart enough to see that the files are only moving22:24
* mordred has a goal to start beating clarkb on code reviews...22:24
* mordred will probably not achieve his goal, but it's good motivation22:24
annegentlefungi: yeah I hope22:24
jeblairmordred: quality is more important than quantity22:25
jeblairjhesketh: thanks for noticing the gaping hole in my new algorithm for zuul.  :)22:27
openstackgerritA change was merged to openstack-infra/config: Add new manual "Cloud Administrator Guide"
fungijeblair: did your change actually stop checking checkForChangesNeededBy() after all?22:29
mordredjeblair: agree22:30
jeblairfungi: no, but it would try to reparent changes without their dependent changes which would end up causing it to flail around because gerrit wouldn't merge them22:30
fungiahh, right22:31
jeblairso it turns out there are some cases where we will have 'dead' items in the queue; items which aren't themselves failing, but can't be merged because their dependent change(s) are failing22:31
mordredjeblair: AH - nice22:31
* fungi needs to go for a quick walk. bbiab22:32
sdaguesorry I missed the conversation earlier on the traces. I'm going to dig into that next week. But it's friday, so time for a beer :)22:33
*** sarob has joined #openstack-infra22:33
mordredsdague: NO - NO BEER FOR YOU - ONLY WORK!!!22:34
mordredspeaking of22:34
mordredI will not be working this weekend22:34
sdagueyou should say that more often22:34
jeblairit's the weekend, no one should be working22:34
mordredas I will be working my other job - Monty Taylor, Lighting Designer22:34
*** olaph has joined #openstack-infra22:34
jeblairmordred: what about the job as Spirit Taster?22:35
mordredjeblair: I'll be doing that later in the week22:35
mordredI think that it would be dangerous to mix Spirit Taster with Stands on Top of Ladder22:35
mordredI have finally read the script though -  and I will say it's quite good22:36
*** w_ has quit IRC22:36
mordredI can totally see how it won the tony a couple of years ago22:36
*** ericw has joined #openstack-infra22:39
anteayaI hope the show is well received22:40
mordredanteaya: the guys I'm working with are top notch - I LOVE working with them because they always do quality, thoughtful work22:41
mordredanteaya: and thank you22:41
mordredfor anyone who happens to be in Austin22:42
*** atiwari has quit IRC22:43
*** nati_ueno_2 has joined #openstack-infra22:46
*** w_ has joined #openstack-infra22:49
*** olaph has quit IRC22:51
*** stevebaker_ has joined #openstack-infra22:51
*** ericw has quit IRC22:51
*** sarob has quit IRC22:52
*** nati_ueno_2 has quit IRC22:55
*** nati_ueno_2 has joined #openstack-infra22:56
*** nati_uen_ has joined #openstack-infra22:58
*** nati_ueno_2 has quit IRC23:00
*** pcrews has quit IRC23:05
*** Ryan_Lane has quit IRC23:07
*** Ryan_Lane has joined #openstack-infra23:07
*** bnemec_away is now known as bnemec23:09
*** vipul is now known as vipul-away23:09
*** nati_uen_ has quit IRC23:09
*** nati_ueno_2 has joined #openstack-infra23:10
*** nati_uen_ has joined #openstack-infra23:11
lifelessmordred: do we have any go-to stats on gerrit? e.g. new patch versions/day?23:12
mordredlifeless: yeah. there are the graphs on zuul status page of course23:14
*** nati_ueno_2 has quit IRC23:14
mordredlifeless: I _believe_ that the other day we did something like 10k jobs run or something23:15
mordredI'm not sure what our patches merged stat is recently, but I konw that it usually hovers between 100-150 patches landed23:15
mordredclarkb: ^^ you got any better go-to numbers?23:15
*** nati_uen_ has quit IRC23:16
*** nati_ueno_2 has joined #openstack-infra23:17
anteayamordred: I hear ya, working with amazing people is awesome23:17
*** stevebaker_ has quit IRC23:17
*** stevebaker_ has joined #openstack-infra23:18
*** vipul-away is now known as vipul23:20
clarkbmordred: graphite is the source of all that23:21
*** nati_ueno_2 has quit IRC23:21
clarkbthe graph at the bottom right of the zuul status page can have its url pasted elswhere and mucked with to show additional data23:21
clarkblifeless: ^23:21
clarkbshow you the number of comments, new patchests, and changes merged23:22
flashgordontook the link and mucked with it to get 4 weeks23:24
lifelessclarkb: thank you!23:24
clarkbI find opening it in firefox makes the url human readable23:24
clarkbwould be nice if chrom* did that too23:24
lifelessanyhow ^ shows some very interesting cyclic behaviour23:25
lifelessaka the work week :)23:25
flashgordoneven better 26 weeks23:25
flashgordonyou can see H323:25
clarkblifeless: and tehre are cycles within milestones, and you can see the larger 6 month cycle23:25
jeblairlifeless, flashgordon: take a /render/ url and s/render/compose/ and you get the graphite composer ui for that graph23:25
jeblair(also, neat trick; s/render/json/ and you get data)23:26
clarkbmordred: I kept that tab open while you were at burning man so I woudln't forget23:28
*** sarob has joined #openstack-infra23:29
*** stevebaker_ has quit IRC23:29
*** stevebaker_ has joined #openstack-infra23:32
*** flaper87 has joined #openstack-infra23:32
flaper87clarkb: ping23:33
clarkbflaper87: pong23:33
flaper87clarkb: hey, any news about moving marconi under openstack/ ?23:33
clarkbflaper87: I haven't had a chance to think about it23:33
flaper87clarkb: kk, whenever you do, could you ping me?23:34
jeblairflaper87: why don't you add it as an agenda item to our meeting?
anteayaflashgordon: nice work on the graphite23:34
clarkbjeblair: fungi mordred ^ flaper87 would like to move marconi under the openstack org as the TC approved incubation (or whatever it is called)23:34
jeblairflaper87: we usually end up discussing such things then anyway, and it would be good if you could attend so we can work out scheduling23:34
jeblairflaper87: (just go ahead and add it to the wiki page with your irc nick)23:35
flaper87jeblair: clarkb will do. Thanks guys!23:35
flashgordonanteaya: thanks turns it wasn't too bad to tinker with the URL23:36
flaper87jeblair: clarkb just to be clear, AFAIU, everything will be reneamed in the database, so, no need to approve all patches (or block) before that happens, right ?23:36
anteayaflashgordon: it looks really pretty23:36
flaper87I'd like to prepare things in advance23:36
jeblairflaper87: correct23:36
flaper87jeblair: awesome, already added it to the agenda23:37
Alex_Gaynorjeblair: can I bribe you to review :)23:37
clarkbflaper87: the only major thing that changes are your git remotes23:37
clarkbeverything else should shift over nicely23:37
flaper87clarkb: got it! Thanks again23:38
clarkblifeless: wrong window?23:38
lifelessclarkb: what is ?23:38
clarkblifeless: there is a way to generate graphs from puppet,23:38
lifelessoh yes23:38
flashgordonanteaya: seeing the H3 bump is pretty neat23:38
anteayabut of a bump, yeah23:39
flashgordonespecailly the review spike23:39
anteayano kidding23:39
clarkblifeless: I didn't realize it had to be done on the agent side, but you can get graphs of the node configs23:39
jeblairAlex_Gaynor: sure, it's a friday afternoon, but hey, flaper87 is around and he knows where to find you if it breaks something.  :)23:39
Alex_Gaynorjeblair: it's cool, they pass :)23:39
clarkbAlex_Gaynor: out of curiousity how many of these just work vs need to be updated?23:40
clarkbAlex_Gaynor: I saw that hacking needed one patch23:40
lifelessclarkb: thats not what I meant23:40
clarkblifeless: what sort of graph are you looking for?23:40
Alex_Gaynorclarkb: the hacking patch I uploaded was for py3k23:40
lifelessclarkb: I meant a graph where the nodes are ci-puppetmaster, puppet-dashboard, zuul etc23:40
openstackgerritA change was merged to openstack-infra/config: Run the marconiclient tests under PyPy
lifelessclarkb: and the edges are 'uses service X'23:41
Alex_Gaynorclarkb: althe clients have been "just work so far", swift needed some help, I think marconi just worked /cc flaper8723:41
jeblairflashgordon: re  what selection criteria did you use for the projects you added it to?23:41
clarkblifeless: for that I don't think so. puppet itself isn't aware of the relationships between nodes as we use it23:41
flaper87Alex_Gaynor: yup, no changes needed to marconi, it just worked23:41
flaper87and I was very happy to get rid of eventlets dependency23:41
lifelessclarkb: given this is puppet's well known heel, I expected no :). But I thought i would ask :)23:41
*** vishious is now known as vishy23:42
clarkbconverting subunit to html is now under 30 seconds for neutron tests
clarkbAlex_Gaynor: lifeless ^ we are converting to v2 before creating the html file23:44
lifelessclarkb: cool23:44
*** hemna is now known as hemnafk23:44
*** CaptTofu has joined #openstack-infra23:45
Alex_Gaynorclarkb: think of all the nature we save with that un-wasted time :)23:46
clarkbAlex_Gaynor: I like to think of it as time to run MOAR tests :)23:48
*** stevebaker_ has quit IRC23:50
*** stevebaker_ has joined #openstack-infra23:53

Generated by 2.14.0 by Marius Gedminas - find it at!