Thursday, 2017-07-27

openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Cleanup pipeline requirements  https://review.openstack.org/48761800:00
*** dingyichen has joined #openstack-infra00:06
*** jaypipes has quit IRC00:09
*** esberglu has joined #openstack-infra00:14
*** gmann has quit IRC00:15
*** gmann has joined #openstack-infra00:15
*** xarses has quit IRC00:18
*** jamesdenton has quit IRC00:19
*** jamesdenton has joined #openstack-infra00:21
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_CHANGES  https://review.openstack.org/48624500:21
*** xarses has joined #openstack-infra00:21
*** xarses has quit IRC00:22
*** xarses has joined #openstack-infra00:22
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_BRANCH  https://review.openstack.org/48624600:24
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_VOTING and add zuul.voting  https://review.openstack.org/48624700:24
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_URL  https://review.openstack.org/48624900:24
*** sree_ has joined #openstack-infra00:32
*** sree_ is now known as Guest4123400:32
*** psachin has joined #openstack-infra00:36
*** Guest41234 has quit IRC00:37
*** bobh has joined #openstack-infra00:39
*** aeng has quit IRC00:40
*** rama_y_ has joined #openstack-infra00:40
*** rama_y has quit IRC00:40
*** rama_y_ has quit IRC00:43
*** rhallisey has quit IRC00:47
*** aeng has joined #openstack-infra00:58
*** gordc has quit IRC01:00
*** LindaWang has joined #openstack-infra01:01
*** cuongnv has joined #openstack-infra01:02
*** liujiong has joined #openstack-infra01:02
*** dmsimard has quit IRC01:03
*** Sukhdev has quit IRC01:06
*** jamielennox has quit IRC01:08
*** makowals has quit IRC01:08
ianwfungi: https://review.openstack.org/#/c/473534 ... am i misreading?01:10
*** jamielennox has joined #openstack-infra01:15
*** armax has joined #openstack-infra01:16
*** dmsimard has joined #openstack-infra01:17
*** aeng has quit IRC01:17
*** makowals has joined #openstack-infra01:18
*** harlowja has quit IRC01:19
*** jkilpatr has quit IRC01:20
*** aeng has joined #openstack-infra01:23
openstackgerritMerged openstack-infra/puppet-exim master: Add smtp_accept_max and smtp_accept_max_per_host  https://review.openstack.org/48745101:25
openstackgerritMerged openstack-infra/system-config master: Raise smtp_accept_max for openstack_project::lists  https://review.openstack.org/48745201:29
*** rossella__ has quit IRC01:30
*** yamamoto_ has quit IRC01:31
*** bobh has quit IRC01:31
*** rossella__ has joined #openstack-infra01:33
*** gongysh has joined #openstack-infra01:35
*** yamamoto has joined #openstack-infra01:35
*** makowals has quit IRC01:38
*** zhurong has joined #openstack-infra01:38
*** esberglu has quit IRC01:43
*** gongysh has quit IRC01:47
*** thorst has joined #openstack-infra01:49
*** thorst has quit IRC01:49
*** dhajare has joined #openstack-infra01:50
*** dhajare_ has joined #openstack-infra01:50
*** dhajare has quit IRC01:50
*** makowals has joined #openstack-infra01:51
fungiianw: i fully believe your comment, though am not in the headspace to trace back through the parent class to confirm the veracity of it just now02:00
*** lrossetti has quit IRC02:03
*** Apoorva has quit IRC02:03
*** markvoelker has quit IRC02:03
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Add centos-7 experimental jobs for stable/ocata  https://review.openstack.org/48766102:04
*** markvoelker has joined #openstack-infra02:06
*** bobh has joined #openstack-infra02:17
*** jamielennox has quit IRC02:17
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Add centos-7 experimental jobs for stable/ocata  https://review.openstack.org/48766102:19
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Run rabbitmqctl report before saving off rabbitmq logs  https://review.openstack.org/48766402:19
*** bobh has quit IRC02:22
*** shu-mutou-AWAY is now known as shu-mutou02:22
*** jamielennox has joined #openstack-infra02:24
*** bobh has joined #openstack-infra02:24
*** yamahata has quit IRC02:30
*** esberglu has joined #openstack-infra02:38
*** baoli has quit IRC02:38
*** liujiong has quit IRC02:39
*** esberglu has quit IRC02:42
*** liujiong has joined #openstack-infra02:43
*** vhosakot has quit IRC02:43
*** sree_ has joined #openstack-infra02:47
*** sree_ is now known as Guest8159302:47
*** Guest81593 has quit IRC02:51
*** rossella__ has quit IRC02:52
*** rossella__ has joined #openstack-infra02:56
* fungi sighs at the citynet abuse desk03:06
fungithey must have somebody new handling it who doesn't realize that just because we have a server instance with an ip address matching an abuse complaint doesn't mean we had an instance at that address at the time the incident occurred03:07
fungii just researched and responded to them on two abuse complaints they forwarded to us where we had no instances using the corresponding ip addresses at the times in question03:08
clarkbkinda like how rax dns blacklists work03:10
fungiyup03:10
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Run rabbitmqctl report before saving off rabbitmq logs  https://review.openstack.org/48766403:11
fungiunfortunately it takes a good amount of time to correlate our nodepool launcher and deleter debug logs to assemble sufficient evidence for debunking03:11
fungibecause it's entirely possible that someone _might_ have submitted a change which portscanned somebody or attempted to deliver spam03:12
fungiso, you know, we have to make sure to check03:13
*** zhurong has quit IRC03:13
*** bobh has quit IRC03:21
*** thorst has joined #openstack-infra03:22
*** vhosakot has joined #openstack-infra03:26
*** thorst has quit IRC03:30
*** harlowja has joined #openstack-infra03:31
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: External Calendar Sync  https://review.openstack.org/48768303:34
*** psachin has quit IRC03:41
openstackgerritTomasz TrÄ™bski proposed openstack-infra/project-config master: Re-enable gates for monasca-events-api  https://review.openstack.org/48760803:46
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove extra GC debug info  https://review.openstack.org/48762203:48
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_PIPELINE  https://review.openstack.org/48625003:50
*** tuanluong has joined #openstack-infra03:51
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_PROJECT  https://review.openstack.org/48625103:56
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove ZUUL_UUID  https://review.openstack.org/48625203:56
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Run rabbitmqctl report before saving off rabbitmq logs  https://review.openstack.org/48766403:56
*** psachin has joined #openstack-infra03:58
*** vhosakot has quit IRC04:00
*** mriedem has quit IRC04:02
*** makowals has quit IRC04:04
*** nicolasbock has joined #openstack-infra04:07
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Case sensitive label matching  https://review.openstack.org/46994604:09
jeblairfungi: it may be worth writing a little script to automatically pull that out of logs (and of course, if we need to add anything to logs to assist, we can)04:11
fungiyeah, that would make sense04:12
jeblairi mean, it'd be better if *they* had that script, but hey.04:13
fungithere's a bit of correlation to do between uuids and ip addresses on different loglines04:13
fungisince we may want to check by uuid or by ip address plus timeframe04:14
fungihow much of this is likely to change with zuul v3... none at this point right?04:15
jeblairfungi: the log lines are a bit different in nodepool v304:15
fungiin that case i'll make a note to look into it once we have the new log format in production04:16
jeblairfungi: i did a thing in the zuul executor to make sure all of the log lines for a job automatically had the job in in them.  we should do the same in nodepool for the node id04:16
jeblairfungi: then it'll be easy to get all the logs for a node.  and the script can do a simple two step process: ip -> node; node -> start+end times04:16
jeblairlike exigrep does with exim logs04:16
fungiyup04:17
fungistart+end times and nova uuid, since that's useful context for the provider to key on in their logs04:18
*** makowals has joined #openstack-infra04:22
*** rama_y has joined #openstack-infra04:30
*** gongysh has joined #openstack-infra04:31
*** links has joined #openstack-infra04:33
*** zhurong has joined #openstack-infra04:38
*** gouthamr has quit IRC04:38
*** sree_ has joined #openstack-infra04:42
*** sree_ is now known as Guest2458604:42
*** Guest24586 has quit IRC04:42
*** ykarel has joined #openstack-infra04:46
*** rama_y has quit IRC04:47
*** prateek has joined #openstack-infra04:52
*** harlowja has quit IRC04:52
openstackgerritDeepak Mourya proposed openstack-infra/os-loganalyze master: Stop ignoring H904 hacking rule in tox  https://review.openstack.org/46876105:00
*** yamahata has joined #openstack-infra05:01
*** gongysh has quit IRC05:05
*** gongysh has joined #openstack-infra05:10
*** gongysh has quit IRC05:13
*** thorst has joined #openstack-infra05:26
*** sree_ has joined #openstack-infra05:28
*** sree_ is now known as Guest4009305:28
*** armax has quit IRC05:30
*** armax has joined #openstack-infra05:31
*** armax has quit IRC05:31
*** armax has joined #openstack-infra05:32
*** gildub has quit IRC05:32
*** armax has quit IRC05:32
*** armax has joined #openstack-infra05:33
*** armax has quit IRC05:33
*** thorst has quit IRC05:33
*** gildub has joined #openstack-infra05:37
*** ccamacho has left #openstack-infra05:41
*** hongbin has joined #openstack-infra05:47
*** jascott1 has joined #openstack-infra05:48
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Case sensitive label matching  https://review.openstack.org/46994605:50
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use correct label casing in tests  https://review.openstack.org/48770305:50
*** rossella__ has quit IRC05:52
*** priteau has joined #openstack-infra05:55
*** rossella__ has joined #openstack-infra05:56
*** yuanying has joined #openstack-infra05:57
*** thorst has joined #openstack-infra06:01
yuanyingmasayukig: around?06:05
*** thorst has quit IRC06:05
*** Guest12964 has quit IRC06:05
*** ramishra has quit IRC06:06
*** ramishra has joined #openstack-infra06:08
*** Guest40093 has quit IRC06:09
openstackgerritDirk Mueller proposed openstack-infra/project-config master: Disable running devstack against openSUSE on stable/ocata and older  https://review.openstack.org/48770906:12
*** esberglu has joined #openstack-infra06:16
*** esberglu has quit IRC06:21
yuvalHey all! Has anyone else experiencing "git review -s" failing because it can not download the commit-msg hook?06:22
yuval*is06:22
*** sree_ has joined #openstack-infra06:31
*** sree_ is now known as Guest1946906:31
*** zhurong has quit IRC06:33
*** kzaitsev_ws has joined #openstack-infra06:33
*** rama_y has joined #openstack-infra06:34
*** aeng has quit IRC06:38
openstackgerritMerged openstack-infra/project-config master: Disable swift in OSC py3 job  https://review.openstack.org/48721106:39
*** cshastri has joined #openstack-infra06:40
openstackgerritMerged openstack-infra/project-config master: Change job type for 3nodes job to move to oooq runner  https://review.openstack.org/48559006:41
openstackgerritMerged openstack-infra/project-config master: Disable running devstack against openSUSE on stable/ocata and older  https://review.openstack.org/48770906:42
openstackgerritMerged openstack-infra/project-config master: Add experimental multinode gate for kuryr-k8s  https://review.openstack.org/48734306:42
*** florianf has joined #openstack-infra06:45
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Exclude v2 jobs from master  https://review.openstack.org/48679506:47
*** florianf has quit IRC06:50
*** florianf has joined #openstack-infra06:51
*** priteau has quit IRC06:51
*** markus_z has joined #openstack-infra06:51
*** rossella__ has quit IRC06:52
*** andreas_s has joined #openstack-infra06:53
*** jtomasek_ has joined #openstack-infra06:54
*** rossella__ has joined #openstack-infra06:55
*** kzaitsev_ws has quit IRC06:56
*** flepied has quit IRC06:56
*** dingyichen has quit IRC06:57
*** coolsvap has joined #openstack-infra07:01
masayukigyuanying: hi07:02
yuanyingmasayukig: Hi, I remember you took a photo at OpenStack Days Tokyo. Could you share it with me?07:03
*** hongbin has quit IRC07:04
yuanyingAs you know, my company needs a report which contains picture to decorate07:05
*** dingyichen has joined #openstack-infra07:06
*** Hal has joined #openstack-infra07:06
*** Hal is now known as Guest4156407:07
masayukigyuanying: ok, sure, but why did you tell me on this channel? :-p07:07
yuanyinghehe07:07
masayukigyuanying: ah,07:07
*** makowals has quit IRC07:08
masayukigyuanying: when was your talk? I actually don't have whole of the photos now because the sd-card is not mine but organizer's one..07:09
*** makowals has joined #openstack-infra07:09
openstackgerritMerged openstack-infra/project-config master: aodh: run hbase and mongodb tests only on newton  https://review.openstack.org/48752407:09
yuanyingMaybe, Friday 14:4507:10
yuanyingmasayukig: Is there any plan to share those photo in public?07:10
openstackgerritMerged openstack-infra/project-config master: Add nova service to kuryr gate  https://review.openstack.org/48652507:11
masayukigyuanying: nice, probably I have that.07:11
masayukigyuanying: I don't have like that plan. They aren't mine.07:11
masayukigyuanying: The organaizer will make it public later, probably,07:12
yuanyingmasayukig: I got it07:12
*** rcernin has joined #openstack-infra07:13
openstackgerritMerged openstack-infra/project-config master: Adding more openSUSE 42.3 variants to existing 42.2 jobs  https://review.openstack.org/48449907:14
*** jtomasek_ has quit IRC07:15
openstackgerritMerged openstack-infra/project-config master: Propose patches to os-service-types from service-types-authority  https://review.openstack.org/48440707:15
yuvalHey all! Is anyone else experiencing "git review -s" failing because it can not download the commit-msg hook?07:16
AJaegeryuval: apparently not ;) which repo?07:19
openstackgerritMerged openstack-infra/project-config master: karborclient: Publish Documentation  https://review.openstack.org/48743607:19
openstackgerritMerged openstack-infra/project-config master: Publish python-senlinclient docs  https://review.openstack.org/48725707:19
AJaegerdtantsur, vsaienko, there are a few changes for ironic in project-config, I'd like your guidenance on, see https://review.openstack.org/#/q/owner:agarwalnisha1980%2540gmail.com+status:open+project:openstack-infra/project-config07:22
AJaegerdtantsur|afk, I see you're not around - and vsaienko neither ;( Ok, will continue to ignore them.07:23
*** wolverineav has joined #openstack-infra07:23
yuvalAJaeger: rpm-packaging07:24
*** HeOS has joined #openstack-infra07:25
AJaegeryuval: works fine for me..07:26
yuvalAJaeger: thanks :\07:26
*** alexchadin has joined #openstack-infra07:26
AJaegeryuval: but isn't git-review just adding gerrit as remote? Just copy & edit over from another repo07:27
AJaegeryuval: did you try git review -s --verbose to debug further?07:27
*** namnh has joined #openstack-infra07:27
openstackgerritMerged openstack-infra/project-config master: Publish some zuul repos  https://review.openstack.org/48720307:28
*** flepied has joined #openstack-infra07:28
yuvalAJaeger: nothing useful there07:29
yuvalAJaeger: might be because it was cloned using https and not ssh07:30
yuvalAJaeger: that was it07:31
AJaegeryuval: yeah, https explains it - you need to set up https access for that07:32
*** jtomasek_ has joined #openstack-infra07:32
*** ccamacho has joined #openstack-infra07:33
*** shu-mutou has quit IRC07:38
*** shu-mutou has joined #openstack-infra07:38
*** [HeOS] has joined #openstack-infra07:41
*** HeOS has quit IRC07:41
*** wolverineav has quit IRC07:45
*** ralonsoh has joined #openstack-infra07:46
*** jtomasek has quit IRC07:46
*** jtomasek_ is now known as jtomasek07:48
*** egonzalez has joined #openstack-infra07:48
*** ralonsoh has quit IRC07:52
*** ralonsoh has joined #openstack-infra07:52
*** jaosorior has joined #openstack-infra07:52
*** hashar has joined #openstack-infra07:59
*** thorst has joined #openstack-infra08:02
*** gildub has quit IRC08:04
*** esberglu has joined #openstack-infra08:04
*** dizquierdo has joined #openstack-infra08:04
*** thorst has quit IRC08:06
*** esberglu has quit IRC08:08
*** markmcd has quit IRC08:10
*** e0ne has joined #openstack-infra08:11
*** markmcd has joined #openstack-infra08:13
*** jascott1 has quit IRC08:14
*** jascott1 has joined #openstack-infra08:15
*** lucas-afk is now known as lucasagomes08:18
*** jascott1 has quit IRC08:20
*** claudiub has joined #openstack-infra08:21
*** efoley has joined #openstack-infra08:28
mrmartinmorgen08:30
mrmartinAJaeger hi, do you know who can I find the url of the release job logs for this patch: https://review.openstack.org/#/c/487508/ ?08:31
*** derekh has joined #openstack-infra08:31
*** electrofelix has joined #openstack-infra08:31
*** dpawlik has quit IRC08:32
*** danpawlik has joined #openstack-infra08:34
*** gildub has joined #openstack-infra08:36
rm_workdid review.openstack.org go down or is it just me <_<08:36
*** gongysh has joined #openstack-infra08:36
mrmartinrm_work: working well here08:38
*** e0ne has quit IRC08:40
*** e0ne has joined #openstack-infra08:42
*** dtantsur|afk is now known as dtantsur08:44
dtantsurhi AJaeger, I'll put them on my list, thanks!08:44
rm_workhmmm08:46
rm_workIRC works...08:46
rm_workother websites work...08:47
rm_workprobably something with my VPN :/08:47
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Enable centos-7 jobs for stable/ocata  https://review.openstack.org/48776308:49
*** gildub has quit IRC08:51
*** dingyichen has quit IRC08:52
AJaegermrmartin: https://docs.openstack.org/infra/manual/developers.html#code-review explains how to find the URL08:53
mrmartinthanks!08:53
*** e0ne has quit IRC08:54
*** e0ne has joined #openstack-infra08:54
rm_workyep GD VPN stopped connecting to review.os.org <_<08:54
*** priteau has joined #openstack-infra08:55
AJaegerdtantsur: thanks! Note that I will wait with a +2 review these until I see you or vsaienko +1 these first...08:58
dtantsurAJaeger: sure, yeah. sorry for losing these from my radar08:58
*** priteau has quit IRC08:59
*** priteau has joined #openstack-infra08:59
AJaegerdtantsur: no worries - I was not sure whether this was a single effort or whether it was team driven...09:00
*** yamahata has quit IRC09:01
openstackgerritMerged openstack-infra/project-config master: Re-enable gates for monasca-events-api  https://review.openstack.org/48760809:07
*** yolanda has quit IRC09:11
openstackgerritMerged openstack-infra/project-config master: networking-midonet: Exclude v2 jobs from master  https://review.openstack.org/48679509:12
*** yolanda has joined #openstack-infra09:12
openstackgerritMerged openstack-infra/project-config master: Enable neutron segment feature in OSC jobs  https://review.openstack.org/48698309:13
openstackgerritMerged openstack-infra/project-config master: new-project: vmware-nsx-tempest-plugin  https://review.openstack.org/48499009:13
*** sambetts|afk is now known as sambetts09:14
openstackgerritMerged openstack-infra/project-config master: Add barbican-tempest-plugin tests to Glance  https://review.openstack.org/48678109:16
*** kjackal_ has joined #openstack-infra09:19
*** shu-mutou is now known as shu-mutou-AWAY09:20
openstackgerritMike Fedosin proposed openstack-infra/project-config master: Glare - Zuul/Jenkins update  https://review.openstack.org/48641809:21
*** wolverineav has joined #openstack-infra09:23
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Add centos-7 experimental jobs for stable/ocata  https://review.openstack.org/48766109:27
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Enable centos-7 jobs for stable/ocata  https://review.openstack.org/48776309:27
*** wolverineav has quit IRC09:27
*** e0ne has quit IRC09:32
kklimondawith zuul, can I define a relation between checks, so some of them (like building binary packages) run before others?09:37
*** e0ne has joined #openstack-infra09:37
AJaegerkklimonda: why do you want this?09:39
kklimondaAJaeger: I want to build a set of binary packages, and then test them against a number of OS releases.09:40
*** wolverineav has joined #openstack-infra09:42
AJaegerkklimonda: See change 476989 on how this could be done, but this needs careful design.09:44
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/48779409:44
AJaegerkklimonda: sorry, no time for further discussion right now...09:44
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Allow loading additional variables file for site config  https://review.openstack.org/44773409:50
*** dtantsur is now known as dtantsur|bbl09:51
*** esberglu has joined #openstack-infra09:52
*** masber has quit IRC09:55
*** sdague has joined #openstack-infra09:55
*** martinkopec has joined #openstack-infra09:56
*** esberglu has quit IRC09:57
*** alexchadin has quit IRC09:57
*** cuongnv has quit IRC10:00
*** yamamoto has quit IRC10:00
*** makowals has quit IRC10:02
*** thorst has joined #openstack-infra10:03
*** makowals has joined #openstack-infra10:04
kklimondaAJaeger: thanks, I'll take a look at how kolla tests are done in general, perhaps talk to the kolla folks too.10:06
*** thorst has quit IRC10:07
*** alexchadin has joined #openstack-infra10:11
*** sshnaidm|afk is now known as sshnaidm10:13
*** dizquierdo has quit IRC10:16
*** katkapilatova has joined #openstack-infra10:23
*** thorst has joined #openstack-infra10:24
*** rlandy has joined #openstack-infra10:28
*** thorst has quit IRC10:28
*** yamamoto has joined #openstack-infra10:31
*** wfoster has quit IRC10:36
*** rook has quit IRC10:37
*** lucasagomes has quit IRC10:37
*** thorst has joined #openstack-infra10:47
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add settings for OVB in RDO Cloud  https://review.openstack.org/48095010:47
*** thorst has quit IRC10:51
*** gildub has joined #openstack-infra11:00
*** dizquierdo has joined #openstack-infra11:03
*** dizquierdo_ has joined #openstack-infra11:06
*** dizquierdo has quit IRC11:08
*** jkilpatr has joined #openstack-infra11:10
*** liujiong has quit IRC11:13
*** [HeOS] has quit IRC11:13
*** wfoster has joined #openstack-infra11:15
*** lucasagomes has joined #openstack-infra11:16
*** kazsh has quit IRC11:18
*** rook has joined #openstack-infra11:18
*** rook is now known as Guest4009811:19
*** kazsh has joined #openstack-infra11:19
*** namnh has quit IRC11:20
*** alexchadin has quit IRC11:20
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574711:21
*** [HeOS] has joined #openstack-infra11:31
*** ldnunes has joined #openstack-infra11:37
*** esberglu has joined #openstack-infra11:40
*** kgiusti has quit IRC11:41
*** jamesden_ has joined #openstack-infra11:44
*** esberglu has quit IRC11:44
*** jamesdenton has quit IRC11:45
*** tuanluong has quit IRC11:47
*** apuimedo has quit IRC11:49
*** thorst has joined #openstack-infra11:54
*** baoli has joined #openstack-infra11:56
*** jokke_ has joined #openstack-infra11:58
*** kjackal_ has quit IRC12:00
*** Guest40098 is now known as rook12:01
*** apuimedo has joined #openstack-infra12:03
*** trown|outtypewww is now known as trown12:05
*** gildub has quit IRC12:06
*** kjackal_ has joined #openstack-infra12:10
*** _nick is now known as yankcrime12:11
*** jcoufal has joined #openstack-infra12:13
*** yamamoto has quit IRC12:14
*** sarlango has quit IRC12:21
*** dizquierdo_ has quit IRC12:24
*** gongysh has quit IRC12:24
*** psachin has quit IRC12:26
*** mriedem has joined #openstack-infra12:27
*** bobh has joined #openstack-infra12:28
*** jcoufal_ has joined #openstack-infra12:28
*** jcoufal has quit IRC12:31
*** gordc has joined #openstack-infra12:34
*** Guest19469 has quit IRC12:35
*** [HeOS] has quit IRC12:36
*** bobh has quit IRC12:37
*** dizquierdo_ has joined #openstack-infra12:37
*** lewo has quit IRC12:39
*** kgiusti has joined #openstack-infra12:39
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Run rabbitmqctl report before saving off rabbitmq logs  https://review.openstack.org/48766412:41
openstackgerritMatt Riedemann proposed openstack-infra/devstack-gate master: Fix existence check for $BASE/logs/rabbitmq/ in cleanup_host  https://review.openstack.org/48782912:41
*** apuimedo has quit IRC12:42
*** apuimedo has joined #openstack-infra12:42
fungiinfra-root: heads up, i'm not around much today; between friends visiting and now some incident with the lines coming over the bridge i'm without broadband internet access for who knows how long but i'll try to check in a few times via my backup wireless modem12:45
smcginnisfungi: Your ISP is trying to tell you something.12:46
fungiapparently12:46
fungisame incident also knocked out power for the next island south of us12:47
TheJuliafun :(12:47
fungichatter is could be days before they get it restored12:47
fungii'll gladly take flaky wireless network access over days without power12:48
*** askb has quit IRC12:50
*** sree has joined #openstack-infra12:50
openstackgerritRob Cresswell proposed openstack-infra/project-config master: Update Horizon & D_O_A jobs  https://review.openstack.org/48783412:51
*** links has quit IRC12:52
*** trown is now known as trown|brb12:55
*** sree has quit IRC12:55
openstackgerritRob Cresswell proposed openstack-infra/project-config master: Update Horizon & D_O_A jobs  https://review.openstack.org/48783412:55
AJaegerodyssey4me: could you review https://review.openstack.org/#/c/486602/ , please?12:57
odyssey4mesure, looking12:57
AJaegerthanks12:57
*** trown|brb is now known as trown13:00
*** bh526r has joined #openstack-infra13:02
*** gcb has quit IRC13:02
*** gcb has joined #openstack-infra13:04
*** rhallisey has joined #openstack-infra13:07
*** [HeOS] has joined #openstack-infra13:11
*** yamamoto has joined #openstack-infra13:14
*** dhajare_ has quit IRC13:15
*** LindaWang has quit IRC13:16
*** spzala has joined #openstack-infra13:16
*** Goneri has joined #openstack-infra13:19
*** yamamoto has quit IRC13:21
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Describe Zuul v3 job naming  https://review.openstack.org/48784813:23
*** esberglu has joined #openstack-infra13:23
AJaegerjeblair, pabelanger, mordred, I'll abandon my Zuul v3 job naming spec now, here's current proposal ^13:24
*** esberglu has quit IRC13:24
*** esberglu has joined #openstack-infra13:24
*** alexchadin has joined #openstack-infra13:25
*** eharney has joined #openstack-infra13:28
*** LindaWang has joined #openstack-infra13:30
*** gouthamr has joined #openstack-infra13:31
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Collect logging information into ara callback  https://review.openstack.org/48785313:32
*** alexchadin has quit IRC13:35
*** liujiong has joined #openstack-infra13:36
*** adarazs is now known as adarazs_brb13:37
*** bobh has joined #openstack-infra13:38
*** bobh has quit IRC13:43
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Collect logging information into ara callback  https://review.openstack.org/48785313:43
openstackgerritSpyros Trigazis (strigazi) proposed openstack-infra/project-config master: [magnum] make legacy swarm -nv  https://review.openstack.org/48785813:47
*** spzala has quit IRC13:49
*** jaypipes has joined #openstack-infra13:50
*** spzala has joined #openstack-infra13:50
*** spzala has quit IRC13:54
*** adarazs_brb is now known as adarazs13:55
*** jcoufal has joined #openstack-infra13:56
*** baoli has quit IRC13:57
*** jcoufal_ has quit IRC13:59
*** jcoufal_ has joined #openstack-infra14:00
*** baoli has joined #openstack-infra14:02
*** jcoufal__ has joined #openstack-infra14:02
*** jcoufal has quit IRC14:03
*** jcoufal_ has quit IRC14:05
*** prateek has quit IRC14:06
*** rwsu has quit IRC14:10
*** kjackal_ has quit IRC14:12
*** sree has joined #openstack-infra14:14
*** brucet has joined #openstack-infra14:15
*** jascott1 has joined #openstack-infra14:17
*** rbrndt has joined #openstack-infra14:18
*** sree has quit IRC14:18
*** bobh has joined #openstack-infra14:18
strigaziAJaeger fungi can you have a look? ^^14:20
*** spzala has joined #openstack-infra14:21
*** jascott1 has quit IRC14:22
openstackgerritMarton Kiss proposed openstack-infra/groups master: Remove patchset #1805690 from drupal core  https://review.openstack.org/48786814:22
*** spzala_ has joined #openstack-infra14:23
AJaegerstrigazi: it's always good if a commit message explains not only what you do but also why14:23
*** jamesden_ has quit IRC14:23
strigaziAJaeger ok14:24
*** jamesdenton has joined #openstack-infra14:25
*** spzala has quit IRC14:25
*** funzo has quit IRC14:27
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test  https://review.openstack.org/47574714:27
openstackgerritSpyros Trigazis (strigazi) proposed openstack-infra/project-config master: [magnum] make legacy swarm -nv  https://review.openstack.org/48785814:30
*** marst has joined #openstack-infra14:33
openstackgerritMerged openstack-infra/groups master: Remove patchset #1805690 from drupal core  https://review.openstack.org/48786814:34
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add IPMI validation before undercloud deploy  https://review.openstack.org/48446214:34
*** ykarel has quit IRC14:38
*** dtantsur|bbl is now known as dtantsur14:39
*** armax has joined #openstack-infra14:39
openstackgerritBradley Jones proposed openstack-infra/project-config master: Add gate for testing microservice ansible deployment  https://review.openstack.org/48787514:39
*** jamesdenton has quit IRC14:39
*** jamesdenton has joined #openstack-infra14:40
*** xarses_ has joined #openstack-infra14:41
*** hashar has quit IRC14:41
ttxHey infra-core, the release team needs some urgent review love on https://review.openstack.org/#/c/483274/ before we start mass-creating stable/pike branches14:41
ttxWe pinged sdague and mtreinish already14:41
sdagueAJaeger: https://review.openstack.org/#/c/487863/ - it's an auto generated xml file that crashes browsers to review it14:43
sdagueis there any reason it's not auto approved?14:43
dhellmannsdague : I have Plans(tm) to change the way we generate sitemaps during queens so we don't need to review them at all14:43
*** andreww has joined #openstack-infra14:44
*** andreww has quit IRC14:45
*** jcoufal has joined #openstack-infra14:45
*** xarses_ has quit IRC14:46
fungittx: dhellmann: looking now14:46
*** andreww has joined #openstack-infra14:47
*** jcoufal__ has quit IRC14:48
fungilgtm14:48
*** cshastri has quit IRC14:48
fungishould probably get periodic stable/pike jobs added soon as well14:48
*** sree has joined #openstack-infra14:49
sdaguedhellmann: well regardless, it would be good to get that out there so that all the links aren't 404s14:52
sdaguewhich they currently are14:52
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: [DNM] larger LVM testing  https://review.openstack.org/48686414:52
dhellmannsdague : oh, sure, I just meant that it would address the issue of auto-approving that you mentioned14:52
AJaegersdague: we updated the sitemap a week ago, so this is only a "minor" update, not everything is broken - only a lot is moving around right now...14:53
*** Swami has joined #openstack-infra14:53
*** jcoufal_ has joined #openstack-infra14:54
sdagueAJaeger: ok, it would be good to be updating this more frequently then14:56
sdaguebecause a week of 404s from google ends up really mudying things up14:56
*** jcoufal has quit IRC14:56
AJaegersdague: we had done this in the past only once with every release.14:59
AJaegerBut yeah, right now we need to do it more often...14:59
sdagueAJaeger: sure, but right now we're sending a lot of people looking for answers into a black hole :)14:59
AJaegersdague: dhellmann has an idea for a superior solution (update with each publish)...14:59
sdaguehttps://docs.openstack.org/nova/latest/cells.html ... very not good14:59
sdaguesure, superior is fine for the future.  :)15:00
sdaguebut this is about the now. Mostly I'm just not understanding why https://review.openstack.org/#/c/487863/ is not +Aed already15:00
*** brucet has quit IRC15:02
*** rcernin has quit IRC15:03
*** links has joined #openstack-infra15:04
*** baoli has quit IRC15:06
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: TLS-everywhere: Explicitly set overcloud_domain_name  https://review.openstack.org/48789915:06
openstackgerritRob Cresswell proposed openstack-infra/project-config master: Update Horizon & D_O_A jobs  https://review.openstack.org/48783415:06
*** links has quit IRC15:07
*** annegentle has joined #openstack-infra15:07
*** LindaWang has quit IRC15:07
*** baoli has joined #openstack-infra15:09
*** jamesdenton has quit IRC15:13
*** dmsimard is now known as dmsimard|afk15:14
*** jamesdenton has joined #openstack-infra15:15
*** waynr has quit IRC15:15
*** annegentle has quit IRC15:16
*** rama_y has quit IRC15:16
*** annegentle has joined #openstack-infra15:17
*** jcoufal has joined #openstack-infra15:18
*** [HeOS] has quit IRC15:19
*** wolverineav has quit IRC15:19
*** jamesden_ has joined #openstack-infra15:20
*** jamesdenton has quit IRC15:20
*** [HeOS] has joined #openstack-infra15:20
*** annegentle has quit IRC15:21
*** jcoufal_ has quit IRC15:21
*** sree has quit IRC15:23
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add gtest-org/gtest project to zuulv3.o.o  https://review.openstack.org/48790515:26
pabelangerjeblair: mordred: I believe ^ is the first step to adding github project into zuulv315:26
*** camunoz has joined #openstack-infra15:26
pabelangerI am not sure if the github integration is setup on gtest-org however15:27
pabelangernot sure where I would check that15:27
*** wolverineav has joined #openstack-infra15:27
*** mat128 has joined #openstack-infra15:29
*** andreas_s has quit IRC15:29
*** skelso has joined #openstack-infra15:29
*** katkapilatova has left #openstack-infra15:30
*** annegentle has joined #openstack-infra15:30
*** jcoufal_ has joined #openstack-infra15:31
*** martinkopec has quit IRC15:32
*** psachin has joined #openstack-infra15:32
mrmartinfungi: hi, as puppetboard.o.o is not working, where can I see the logs of the last puppet run for a host?15:33
*** camunoz has quit IRC15:33
*** jcoufal has quit IRC15:33
mordredpabelanger: I can do that real quick15:33
*** wolverineav has quit IRC15:34
*** baoli has quit IRC15:34
*** baoli has joined #openstack-infra15:35
*** gyee has joined #openstack-infra15:35
*** claudiub has quit IRC15:37
mordredpabelanger: "Alright, OpenStack Zuul was installed on the @gtest-org account." - I went ahead and just added it to the whole org15:37
pabelangermordred: great15:37
clarkbI'm not sure if everyone saw, but yesterday I got zuul working with gerrit 2.13 on review-dev. It did require the use of tobiash's case sensitivity change. Also confirmed that the gerrit events will emit vote values even if already set to that value. This means recheck on a verified +1 will result in gating15:37
fungimrmartin: puppetboard got deprecated a couple years ago when we ceased being able to update it for modern puppet, and then we switched to ansible for a bunch of the higher-level parts... the hope is that we can use ara for that though for the moment the only place we publicly expose ansible/puppet events is via mqtt from firehose.openstack.org (and the ansible events there might still be broken until15:37
fungihttps://review.openstack.org/485275 gets approved)15:37
mordredpabelanger: we should probably add a doc somewhere on this - I can make a patch - basic process is to go to https://github.com/apps/openstack-zuul then click some buttons15:37
*** rossella__ has quit IRC15:38
*** annegentle has quit IRC15:38
*** Douhet has quit IRC15:39
pabelangermordred: Ah, I see. Cool15:39
*** Douhet has joined #openstack-infra15:40
*** rossella__ has joined #openstack-infra15:41
mrmartinfungi: will be great to remove the entire puppetboard.o.o instance / domain.15:42
mrmartinwill / would be15:42
fungimrmartin: i agree, i think we had agreed to do that a while back but nobody has tackled it yet15:43
pabelangerfungi: mrmartin: https://review.openstack.org/#/c/449167/15:43
pabelangershould rebase to get out of merge conflict15:43
fungioh, excellent15:43
fungithanks pabelanger15:43
pabelangerlet me clean it up quickly15:44
*** sambetts is now known as sambetts|afk15:45
*** e0ne has quit IRC15:47
*** camunoz has joined #openstack-infra15:48
*** hongbin has joined #openstack-infra15:48
*** spzala_ has quit IRC15:49
*** liujiong has quit IRC15:49
*** baoli has quit IRC15:49
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove puppetdb / puppetboard server  https://review.openstack.org/44916715:51
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: [DNM] larger LVM testing  https://review.openstack.org/48686415:51
*** Swami has quit IRC15:54
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix race in test_periodic_override  https://review.openstack.org/48791715:54
*** efoley has quit IRC15:56
*** baoli has joined #openstack-infra15:57
*** rama_y has joined #openstack-infra16:00
*** egonzalez has quit IRC16:02
openstackgerritDaniel Mellado proposed openstack-infra/project-config master: Enable Octavia Variant  https://review.openstack.org/48792816:08
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Simplify run tox task  https://review.openstack.org/48755116:08
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't request empty nodesets  https://review.openstack.org/48724316:10
*** spzala has joined #openstack-infra16:11
clarkbfungi: sdague https://review.openstack.org/#/c/486676/ is a d-g change to address an error we sometimes see in rackspace. ianw has already +2'd16:13
sdaguebam!16:14
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Fix race in test_periodic_override  https://review.openstack.org/48791716:14
clarkbsdague: tyty16:14
*** spzala has quit IRC16:15
pabelangermordred: care to review: https://review.openstack.org/#/c/487905/ re: gtest-org patch16:16
clarkbpabelanger: not sure if it affects you are not but I've already got a zuul.yaml in there and have used it as a trusted project on my gerrit 2.13 test setup.16:18
clarkbpabelanger: it is configured to run check and gate noop jobs. I think you can delete that config or replace it at this point though as I got review-dev and zuulv3 working yesterday16:18
pabelangerclarkb: okay, I can use another repo if needed too16:18
mordredpabelanger: yah - let's use a different repo so we don't step on clarkb16:19
pabelanger++16:19
mordredpabelanger: in fact ... one sec16:19
clarkbsorry I picked gtest bceause I already had it cloned and was using it for other testing when I upgraded gerrit to 2.1316:19
pabelangerclarkb: np! We can update16:20
mordredpabelanger: https://github.com/gtest-org/ansible how about we use that, since the main thing we want to initial test is the shade-ansible test anyway16:20
mordredpabelanger: that way we can write non-throwaway content but not bother anybody16:20
pabelangermordred: Ah, I considered doing that :D16:20
mordredpabelanger: :)16:20
*** ramishra has quit IRC16:21
*** funzo has joined #openstack-infra16:21
*** funzo has quit IRC16:21
*** funzo has joined #openstack-infra16:21
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add gtest-org/ansible project to zuulv3.o.o  https://review.openstack.org/48790516:21
*** mat128 has quit IRC16:24
openstackgerritDaniel Mellado proposed openstack-infra/project-config master: Enable Octavia Variant  https://review.openstack.org/48792816:24
*** eumel8 has joined #openstack-infra16:25
openstackgerritFrank Kloeker proposed openstack-infra/system-config master: Installation of i18n-checksite.openstack.org  https://review.openstack.org/48793616:25
eumel8don't know if this right16:26
*** ccamacho has left #openstack-infra16:27
*** ralonsoh has quit IRC16:28
*** spzala has joined #openstack-infra16:29
*** psachin has quit IRC16:32
*** spzala has quit IRC16:33
ianychoieumel8, then openstack-infra/puppet-translation_checksite is not needed if the review ^ works?16:33
eumel8yes, it's for the cloud museum16:34
fungii'm sure we have a fine spot in the display case for that16:37
eumel8I've added some reviewers yet. Maybe Andy has a better idea to implement that16:39
*** vhosakot has joined #openstack-infra16:43
*** jcoufal has joined #openstack-infra16:46
*** jcoufal_ has quit IRC16:49
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Document Zuul v3 GitHub integration  https://review.openstack.org/48794116:54
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Fix a few docs table of contents references  https://review.openstack.org/48794216:54
mordredpabelanger: ^^ there's the missing docs on the setup16:56
*** Apoorva has joined #openstack-infra16:59
*** baoli has quit IRC16:59
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Switch to use tox_chdir for tox-py35-on-zuul job  https://review.openstack.org/48794817:00
*** bh526r has quit IRC17:00
pabelangermordred: great17:01
*** hyakuhei has quit IRC17:02
*** hyakuhei has joined #openstack-infra17:02
*** hyakuhei has quit IRC17:02
*** hyakuhei has joined #openstack-infra17:02
*** spzala has joined #openstack-infra17:02
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: Switch to use tox_chdir for tox-py35-on-zuul job  https://review.openstack.org/48794817:02
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Simplify run tox task  https://review.openstack.org/48755117:02
openstackgerritMerged openstack-infra/devstack-gate master: Set up stable/pike  https://review.openstack.org/48327417:03
*** derekh has quit IRC17:03
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Start storing results with the SQL reporter  https://review.openstack.org/48795017:03
openstackgerritFrank Kloeker proposed openstack-infra/system-config master: Installation of i18n-checksite.openstack.org  https://review.openstack.org/48793617:03
*** skelso has quit IRC17:04
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Case sensitive label matching  https://review.openstack.org/46994617:05
openstackgerritKen Giusti proposed openstack-infra/project-config master: Temporarily disable voting on non-rabbit oslo.messaging jobs  https://review.openstack.org/48795117:05
*** baoli has joined #openstack-infra17:06
*** spzala has quit IRC17:06
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Simplify run tox task  https://review.openstack.org/48755117:07
openstackgerritMerged openstack-infra/project-config master: Add gtest-org/ansible project to zuulv3.o.o  https://review.openstack.org/48790517:07
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Simplify run tox task  https://review.openstack.org/48755117:09
*** harlowja has joined #openstack-infra17:09
*** baoli has quit IRC17:10
*** baoli has joined #openstack-infra17:11
jeblairmordred: nice docs!17:12
*** spzala has joined #openstack-infra17:13
*** xarses_ has joined #openstack-infra17:15
mordredjeblair: thanks! I'm making another followup patch to that ...17:16
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Simplify run tox task  https://review.openstack.org/48755117:16
*** rcernin has joined #openstack-infra17:16
*** spzala has quit IRC17:17
*** skelso has joined #openstack-infra17:17
*** xarses has quit IRC17:18
*** jcoufal_ has joined #openstack-infra17:18
mnaserim getting a bunch of job timeouts for puppet-openstack-integration for puppet modules throughout the day today17:19
mnaseris there a provider that is seeing slowness or are we onto an issue?17:19
*** mat128 has joined #openstack-infra17:20
clarkbmnaser: I'm not aware of any region/provider specific slowness out of the ordinary17:21
clarkbwe do try to track generic timeouts though with e-r17:21
*** jcoufal has quit IRC17:21
clarkb(so there is a query you can click on on the e-r status page to see them and can be used to see if a region is particularly bad)17:22
mnaserfor example just today i had these two - http://logs.openstack.org/27/487927/1/check/gate-puppet-openstack-integration-4-scenario002-tempest-centos-7/47712fb/console.html and http://logs.openstack.org/17/487617/2/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/574e3ec/17:22
mnaserseem to be mostly puppet-openstack jobs so i wonder if there is something there17:23
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Use correct label casing in tests  https://review.openstack.org/48770317:23
*** dtantsur is now known as dtantsur|afk17:24
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Capitalize gerrit labels in zuulv3 config  https://review.openstack.org/48795517:26
*** Douhet has quit IRC17:26
*** Douhet has joined #openstack-infra17:26
*** Apoorva_ has joined #openstack-infra17:26
*** spzala has joined #openstack-infra17:26
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Start storing results with the SQL reporter  https://review.openstack.org/48795017:27
*** mat128 has quit IRC17:29
*** Apoorva has quit IRC17:29
*** camunoz has quit IRC17:29
*** jamesden_ has quit IRC17:29
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: Setup tox_chdir for tox-py35-on-zuul job  https://review.openstack.org/48794817:31
*** jamesdenton has joined #openstack-infra17:31
*** lucasagomes is now known as lucas-afk17:31
*** jcoufal has joined #openstack-infra17:31
*** jascott1 has joined #openstack-infra17:32
*** mat128 has joined #openstack-infra17:34
*** jcoufal_ has quit IRC17:34
*** skelso has quit IRC17:34
*** rbrndt has quit IRC17:36
*** skelso has joined #openstack-infra17:37
*** SumitNaiksatam has joined #openstack-infra17:38
*** Apoorva_ has quit IRC17:39
*** yamahata has joined #openstack-infra17:40
AJaegermordred, jeblair, pabelanger, could you put the job naming change on your review queue, please? https://review.openstack.org/48784817:40
jeblairAJaeger: wow, infra-manual has changed a lot.  there are a lot more than 3 guides.17:41
*** ijw has joined #openstack-infra17:41
*** Apoorva has joined #openstack-infra17:41
*** camunoz has joined #openstack-infra17:42
fungiwe might could stand to reorganize/split the infra-manual yes17:42
*** sshnaidm is now known as sshnaidm|off17:42
*** notmyname has quit IRC17:43
*** electrofelix has quit IRC17:43
fungiit grew some "howto" type procedural docs in addition to the role-specific references17:43
jeblairi'd suggest combine.  at least, the thing i was going for was that there would be 3 guides, targeted to audience: developer, reviewer, driver.17:43
jeblairi mean, if folks have another idea of how to organize it, i won't stand in the way.  i only ask that we *have* an idea, and stick with it as much as possible.  :)17:44
fungiso to take an early example, which audience is project creation most suited to? developer i guess?17:44
jeblairso i'd suggest moving the creators guide into drivers, sandbox into developers, and test environment into drivers17:44
jeblairfungi: driver17:44
*** notmyname has joined #openstack-infra17:44
dhellmannis there some way to see the logs of a doc gate/check job instead of the built docs?17:44
fungioh, right driver17:44
jeblairdhellmann: go up one dir17:44
dhellmannthanks17:44
AJaegerdhellmann: replace docs-draft with logs in the URL as well17:45
fungidhellmann: s/docs-draft/logs/ in the url17:45
jeblairfungi: 'driver' was the word intended to convey "someone who helps run the project".  of course, we stopped using that word for some reason17:45
jeblairdhellmann: oh, yeah, sorry don't listen to me17:45
dhellmannbah, sphinx doesn't show the names of static files it is copying17:45
AJaegerdhellmann: but the ansible log does17:46
dhellmanndoes it?17:46
jeblairi might have been thinking of a, erm, "different" system.  :)17:46
fungiwe didn't so much intentionally stop using the term "driver" as it ended up being synonymous with release managers from a gerrit perspective17:46
openstackgerritMerged openstack-infra/project-config master: Capitalize gerrit labels in zuulv3 config  https://review.openstack.org/48795517:46
AJaegerdhellmann: which change you're looking at?17:46
dhellmannhttp://logs.openstack.org/32/487932/2/check/gate-nova-docs-ubuntu-xenial/5b3e784/_zuul_ansible/ansible_log.txt17:46
dhellmannI'm trying to figure out if I've put that .htaccess file in the right place17:46
AJaegerdhellmann: ah, only for the post job...17:47
fungiwe had some fooproject-driver and fooproject-release groups in gerrit, and ended up converting the former to the latter for consistency17:47
AJaegerdhellmann: check "2017-07-27 17:33:11,826"17:47
AJaegerdhellmann: and I don't see .htaccess ;(17:47
dhellmannok17:48
dhellmannthere are dot files, so that's not the problem17:48
dhellmannI must not have set up the file in the build properly, I'll see what happens locally17:48
dhellmannthanks, AJaeger17:48
AJaegerdhellmann: is static configured in the conf.py?17:48
dhellmannAJaeger : yes17:49
jeblairfungi: yeah.  if there's a better word, i'm all for it.  it's a good word if you think about it, but it doesn't immediately call to mind what we're trying to convey.17:49
*** spzala has quit IRC17:50
fungii doubt we made that consolidation with any thoughts of connotation, but rather merely statistics17:50
fungiat least i don't remember much beyond "there are more -release groups in gerrit now"17:51
fungieasy enough to rename groups in gerrit these days. sed -i the acls and script a mass group rename via the api if that's something we want17:52
*** mat128 has quit IRC17:53
*** trown is now known as trown|lunch17:53
jeblairfungi: oh i meant for the docs :)  like, if there's a better name for "Project Driver's Guide" i'm all for it17:55
*** ldnunes has quit IRC17:56
*** SumitNaiksatam has quit IRC17:56
*** SumitNaiksatam has joined #openstack-infra17:56
*** ldnunes has joined #openstack-infra17:57
AJaegerthe project driver's guide as one large page - or should we split it into sub pages?17:57
openstackgerritLin Yang proposed openstack-infra/project-config master: Mark all jobs for python-valenceclient to voting  https://review.openstack.org/48760217:57
fungijeblair: right, and i was talking about if we already have a better term in the docs and want some consistency with our internal implementation details17:57
fungiflip side of the same coin17:57
jeblairgotcha17:57
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Split out and expand zuulv3 docs  https://review.openstack.org/48796217:59
mordredjeblair: ^^ there's another followup for sysadmin docs18:00
jeblairmordred: nice18:04
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Describe Zuul v3 job naming  https://review.openstack.org/48784818:04
AJaegerjeblair: addressed most of your comments but did not merge it yet - so that diff works ^18:05
dhellmannok, I have the .htaccess file copying into place now18:05
clarkbmordred: I've approved that change but left a comment on an item we may awnt to update as we are updating all the things (it comes from the older docs so wasn't going to ding this change for it)18:06
dhellmannfungi, jeblair : I'm trying to address sdague and mriedem's concerns about old urls not working by designing a way for projects to put redirects in their doc trees18:06
dhellmannI have the .htaccess files in place, but it doesn't seem to be redirecting the way I expected18:06
dhellmannhttps://review.openstack.org/#/c/487932/3/doc/source/_extra/.htaccess is the file18:06
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Describe Zuul v3 job naming  https://review.openstack.org/48784818:07
AJaegerjeblair: now merged into the drivers guide ^18:07
dhellmannthe logs show it copying: http://logs.openstack.org/32/487932/3/check/gate-nova-docs-ubuntu-xenial/769977f/_zuul_ansible/ansible_log.txt18:07
clarkbdo we have to enable thinsg on the server to allow them to work as expected? (re htaccess)18:07
dhellmannbut when I try to hit http://docs-draft.openstack.org/32/487932/3/check/gate-nova-docs-ubuntu-xenial/769977f/doc/build/html/redirect-test.html I get a 404 instead of 30118:07
dhellmanndoes the server on docs-draft support redirects?18:07
dhellmannand do we in general support having .htaccess files anywhere other than the root of the web server?18:07
clarkbdhellmann: I don't think we generally allow it anywhere18:08
clarkblooks like allowoverride is the necessary directive?18:08
dhellmannwas that set (or not set) because we don't want to allow it for some reason, or because it  hasn't come up or wasn't useful before?18:09
dhellmann maybe this approach is a bad idea?18:09
clarkbI think its generally off by default for performance reasons18:10
clarkb(there is some overhead to processing htaccess rules18:10
dhellmannthat makes sense18:10
* AJaeger calls it a day and waves good bye18:10
dhellmannAJaeger : o/18:11
sdagueclarkb: the thing you'll find documented is also the performance hit on the fstat of the .htaccess file18:11
sdaguebecause they are dynamically checked and loaded18:11
sdaguebut, that being said, the overhead is probably not a huge deal18:12
clarkbdhellmann: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/static-http-and-https.vhost.erb that is the vhost def18:12
clarkballowoverride is set to none so that explains it18:12
dhellmannclarkb : cool, thanks18:12
dhellmannso then the question becomes, is it ok to turn that on?18:12
dhellmannI should probably write this plan down in more detail so people can consider their answers18:12
clarkbI think it could potentially be abused to redirect you offhost if someone got a malicious change in18:13
sdagueclarkb: it would be governed by normal code review18:14
clarkbsdague: thats true, though the draft site wouldn't be, but I think we care much less on the draft site18:14
dhellmannyeah18:14
clarkbbut thats the only concern I can come up with18:14
sdagueit would probably be a good todo to add some kind of rule checks on .htaccess files for basic sanity at some point as well, but we could move that in after this is in place. I'd rather unstick us now.18:16
dhellmannyep18:16
*** mat128 has joined #openstack-infra18:16
*** [HeOS] has quit IRC18:21
openstackgerritLin Yang proposed openstack-infra/project-config master: Add python and client libray for Rack Scale Design  https://review.openstack.org/48760118:24
jeblairclarkb, sdague, dhellmann: maybe use allowoverridelist and specify exactly the directives (eg "Redirect RedirectMatch") that we want to support ?18:25
dhellmannclarkb : I could use your feedback on http://lists.openstack.org/pipermail/openstack-dev/2017-July/120248.html (is it enough detail, does it make sense, etc.)18:25
dhellmannjeblair : sure18:25
dhellmannjeblair : ^^ ML thread :-)18:25
dhellmann  if there's a way to prevent off-site redirects we could use that directive, too18:26
dhellmannI don't know if such a thing exists18:27
openstackgerritLin Yang proposed openstack-infra/project-config master: Mark all jobs for python-valenceclient to voting  https://review.openstack.org/48760218:28
*** florianf has quit IRC18:28
clarkbdhellmann: digging more docs.o.o and docs-draft use two different vhost templates18:28
dhellmannI don't think we care too much about the draft site18:28
clarkbdhellmann: the one I linked is for docs-draft https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/docs.vhost.erb is docs.o.o18:28
dhellmannI mean, it would be nice to be able to test the redirects, I guess18:29
dhellmannso maybe we do18:29
clarkbya I think we should enable in both places18:29
*** annegentle has joined #openstack-infra18:29
dhellmannyeah, that makes sense now that I think it through18:29
* dhellmann shouldn't make kneejerk decisions18:29
dhellmannok, I've updated my personal notes about that, I can do them both in 1 patch18:30
jeblairdhellmann: re offsite redirs: i don't either.  i think it's okay to set guidelines and generally trust folks to review well.  my suggestion was mostly because htaccess is *immensely* powerful, and i don't want us to end up in a situation where we accidentally open up a vulnerability, or end up using features we don't know about.18:30
dhellmannjeblair : yeah, that totally makes sense. I was trying to combine your suggestion with clarkb's concern to see if we could address it18:31
jeblair*nod*18:31
dhellmannbut I generally agree that a review policy is good, and if we can do something automated later that would be good, too18:31
clarkbI think you want allowoverride fileinfo \n allowoverridelist redirect redirectmatch18:34
*** dizquierdo_ has quit IRC18:34
clarkbin the docs.o.o vhost then in draft you want allowoverride none \n allowoverridelist redirect redirectmatch?18:34
openstackgerritLin Yang proposed openstack-infra/project-config master: Add python and client libray for Rack Scale Design  https://review.openstack.org/48760118:34
clarkbbut ya not seeing a way to further restrict redirect directives18:35
*** ijw_ has joined #openstack-infra18:36
*** rbrndt has joined #openstack-infra18:37
dhellmannthanks clarkb, I'll make a note of those18:37
dhellmannso, next question, how hard is it for me to see a recent list of 404s? :-)18:37
clarkbshouldn't be too hard for an infra root to grep them out of the apache access log18:38
clarkblet me go do that now18:39
dhellmannclarkb : if you could do 301 and 404 at the same time that would be great18:39
*** ijw has quit IRC18:40
*** ijw_ has quit IRC18:41
clarkbdhellmann: operation, url, count a good set of fields?18:41
fungidhellmann: catching up on scrollback (i'm only intermittently around today) but we may need to permit certain .htaccess directives for those subtrees in the apache vhost config? i haven't looked... what behavior are you seeing?18:41
clarkbdhellmann: you don't need agent string or ip addrs or timestamp right?18:41
fungioh, clarkb already said something similar18:42
dhellmannclarkb : timestamp might be useful, but if you're limiting to a range (say the last few days or week? I don't know what's easy for you to access) that could be left out18:42
fungiwe may want to avoid allowing overrides on docs-draft due to the pre-merge nature of what's published there, so as to discourage abuse18:42
dhellmannfungi : yep, clarkb proposed a specific change for me to make, which I will do after I've looked at what those directives mean so I know what I'm proposing :-)18:42
dhellmannyeah, I went back and forth on the docs-draft thing18:43
fungiand again, clarkb said this18:43
dhellmannit would be useful to be able to test the redirects, but it's not that big of a deal18:43
*** hashar has joined #openstack-infra18:43
* fungi is overjoyed at his own apparent redundancy in discussions of this sorty18:43
dhellmannheh18:43
dhellmannthat's the sign of a good team, for sure18:44
*** skelso has quit IRC18:44
fungithe conversation appears to have gone much farther, and i agree with the assertions of others. i'm going back outside now ;)18:44
dhellmannfungi : enjoy!18:44
fungithanks!18:44
openstackgerritMerged openstack-infra/system-config master: Document Zuul v3 GitHub integration  https://review.openstack.org/48794118:44
openstackgerritMonty Taylor proposed openstack-infra/puppet-zuul master: Add PyMySQL python library for SQL Reporter support  https://review.openstack.org/48797418:44
openstackgerritMerged openstack-infra/system-config master: Fix a few docs table of contents references  https://review.openstack.org/48794218:46
openstackgerritMerged openstack-infra/system-config master: Split out and expand zuulv3 docs  https://review.openstack.org/48796218:46
*** skelso has joined #openstack-infra18:47
*** trown|lunch is now known as trown18:48
*** armax has quit IRC18:49
openstackgerritLin Yang proposed openstack-infra/project-config master: Add python and client libray for Rack Scale Design  https://review.openstack.org/48760118:50
*** rossella__ has quit IRC18:53
clarkbdhellmann: there are 2913 404s from today's log do you want ot top 100 or something?18:53
clarkbI guess I can use gist to get a full list18:53
clarkbs/get/share/18:53
dhellmannwowo18:53
dhellmannyeah, maybe a top 10018:53
*** rossella__ has joined #openstack-infra18:54
openstackgerritMonty Taylor proposed openstack-infra/puppet-zuul master: Add PyMySQL python library for SQL Reporter support  https://review.openstack.org/48797418:55
clarkbdhellmann: http://paste.openstack.org/show/616773/ that is 404s18:55
*** camunoz has quit IRC18:55
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Enable SQL support for zuul v3 scheduler  https://review.openstack.org/48797518:55
*** camunoz has joined #openstack-infra18:56
dhellmannew, lots of json and css in there18:56
dhellmannin fact almost all of that is json and css18:56
dhellmannclarkb : can you filter on html files, please?18:57
dhellmannor urls ending in / I guess18:57
*** ijw has joined #openstack-infra18:57
clarkbdhellmann: http://paste.openstack.org/show/616774/ is 301s, ya let me update18:57
dhellmannty, I appreciate the effort18:57
clarkbdhellmann: http://paste.openstack.org/show/616775/ and http://paste.openstack.org/show/616776/ first line shows if it is 404s or 301s19:00
openstackgerritMerged openstack-infra/devstack-gate master: Sync and partprobe after mkpart on ephemeral disk  https://review.openstack.org/48667619:00
dhellmannclarkb : fantastic, thank you19:00
*** ijw has quit IRC19:01
*** dmsimard|afk is now known as dmsimard19:04
openstackgerritOpenStack Proposal Bot proposed openstack-dev/hacking master: Updated from global requirements  https://review.openstack.org/44002919:07
openstackgerritOpenStack Proposal Bot proposed openstack-dev/pbr master: Updated from global requirements  https://review.openstack.org/48798119:07
openstackgerritOpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements  https://review.openstack.org/48594319:07
*** SumitNaiksatam has quit IRC19:07
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements  https://review.openstack.org/48798919:08
*** Apoorva_ has joined #openstack-infra19:09
*** Apoorva has quit IRC19:12
*** xyang1 has joined #openstack-infra19:12
*** rossella__ has quit IRC19:16
*** flepied has quit IRC19:17
*** Apoorva_ has quit IRC19:19
*** rossella__ has joined #openstack-infra19:19
*** Apoorva has joined #openstack-infra19:19
*** eharney_ has joined #openstack-infra19:24
*** eharney has quit IRC19:25
*** eharney_ is now known as eharney19:25
*** jamesdenton has quit IRC19:26
*** jamesdenton has joined #openstack-infra19:26
*** eharney has quit IRC19:30
pabelangerclarkb: fungi: do you know why pip install -U /opt/zuul would _not_ install github3.py using github.com URL in http://git.openstack.org/cgit/openstack-infra/zuul/tree/requirements.txt?h=feature/zuulv3#n5 ?19:30
pabelangerclarkb: fungi: it instead grabs it from pypi19:30
clarkbpabelanger: maybe -U is deciding a version on pypi is neewer and should be updated too? though that would be a pip bug I think19:32
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Use mypy to do static type checking  https://review.openstack.org/48816119:34
pabelangerclarkb: by using -r /opt/zuul/requirements.txt -U /opt/zuul it works as expected19:34
pabelangermordred: Oh, I think we need github settings on ze01.o.o too19:34
pabelangermordred: let me manually apply to see19:35
clarkbpabelanger: in that case it is possibly a pbr bug19:35
mordredpabelanger: do we? weird ... I thought all th eapi interactions were in the driver in the scheduler19:35
clarkbpabelanger: since pbr is what reads in the requirements file19:35
mordredpabelanger: unless it's trying to clone via ssh and there's a key  missing?19:36
*** skelso has quit IRC19:36
pabelangermordred: ya, think so: http://paste.openstack.org/show/616782/19:38
clarkbcode looks the same19:38
pabelangermordred: maybe just needs ssh key, not apt credentials19:38
pabelangerclarkb: k19:38
clarkbcode looks same was meant for #zuul19:38
openstackgerritOpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements  https://review.openstack.org/48594319:43
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Update docs on github connection settings  https://review.openstack.org/48816519:44
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Update docs with correct information about github apps  https://review.openstack.org/48816719:46
* fungi sees ping about pip installing git urls, notices it seems to have been subsequently answered in #zuul, disappears again19:47
mordredpabelanger: "Host key verification failed."19:48
*** nicolasbock has quit IRC19:48
pabelangermordred: ya, working on fix: https://etherpad.openstack.org/p/zuulv3-github current issues19:49
mordredpabelanger: cool19:49
*** [HeOS] has joined #openstack-infra19:49
mordredpabelanger: are you working on puppet for the known_hosts?19:51
*** e0ne has joined #openstack-infra19:51
pabelangermordred: not yet19:53
pabelangermordred: was the SSH key for openstack-zuul added to github?19:53
pabelangerPermission denied (publickey).19:54
pabelangerchecking github settings19:54
openstackgerritOpenStack Proposal Bot proposed openstack/os-client-config master: Updated from global requirements  https://review.openstack.org/48808419:54
openstackgerritDoug Hellmann proposed openstack-infra/system-config master: allow redirects in .htaccess files on the static web servers  https://review.openstack.org/48817219:57
openstackgerritDoug Hellmann proposed openstack-infra/system-config master: allow redirects in .htaccess files on the static web servers  https://review.openstack.org/48817220:00
mordredpabelanger: much easier fix I think20:00
openstackgerritOpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements  https://review.openstack.org/48594320:01
mordredpabelanger: we should remove "git_ssh_key" from the config for that20:01
mordredif we do - it'll clone over https20:01
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Remove ssh key from github connection and add to executor  https://review.openstack.org/48817820:07
mordredpabelanger: ^^20:08
mordredjeblair: ^^ was there a reason we were doing ssh vs. https there? and I'm correct about executors needing connection info right?20:09
pabelangermordred: sure, I can test quickly20:11
openstackgerritOpenStack Proposal Bot proposed openstack/os-client-config master: Updated from global requirements  https://review.openstack.org/48808420:12
*** eharney has joined #openstack-infra20:13
pabelangermordred: yup!20:13
pabelanger2017-07-27 20:13:05,135 DEBUG zuul.Repo: Cloning from https://github.com/gtest-org/ansible to /var/lib/zuul/executor-git/github.com/gtest-org/ansible20:13
*** skelso has joined #openstack-infra20:13
clarkbyou'd only need ssh if pushing right? though https can be pushed too now with github but its clunky like gerrit20:15
pabelangerthink so, and we wouldn't to that on zuulv3.o.o20:16
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Stop double-logging the build id  https://review.openstack.org/48818320:17
openstackgerritMerged openstack-infra/zuul master: Fix passing labels to Gerrit when they are not defined in All-Projects  https://review.openstack.org/42480620:17
*** e0ne has quit IRC20:17
mnaserif i abandon a change, can it be taken over by someone else later?20:18
mnaserthere's a proposed spec that's been sitting in my queue and its not something that i'm working on actively, whats the best way to 'gracefully' leave it open20:18
openstackgerritOpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements  https://review.openstack.org/48594320:19
clarkbmnaser: yes abandoned changes can be restored later by the change owner or any project core20:19
*** jtomasek has quit IRC20:19
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add github connection to check pipeline  https://review.openstack.org/48818620:20
jeblairmordred: looking20:21
*** xyang1 has quit IRC20:21
clarkbpabelanger: review on ^20:21
*** jkilpatr has quit IRC20:21
jeblairmordred: 178 wfm +320:22
pabelangerclarkb: ya, my syntax is wrong also20:22
pabelangerclarkb: looking at docs now20:22
mnaseri wonder what caused a huge spike of test nodes in the past few minutes :x20:22
*** Goneri has quit IRC20:22
jeblairpabelanger: why didn't zuul report the error?20:23
mnaserlooks like it jumped from ~750-1000 to ~1750 almost instantly20:23
pabelangerjeblair: I am not sure20:23
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Stop double-logging the build id  https://review.openstack.org/48818320:23
clarkbmnaser: looks like global requirements updates20:23
mnaseroh fun times20:23
*** makowals has quit IRC20:24
mordredjeblair: mind quick +As on https://review.openstack.org/#/c/488167 and https://review.openstack.org/#/c/487975 which are in the stack with 178?20:25
mordredoh - and https://review.openstack.org/#/c/487974/20:25
mordredthe rabbit hole goes deep today20:26
*** makowals has joined #openstack-infra20:26
jeblairpabelanger: oh, i think i understand.  we don't include pipelines in any kind of dynamic reconfiguration, even the one that we do on trusted projects then throw away.  i think we can change that.20:26
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add github connection to check pipeline  https://review.openstack.org/48818620:26
*** mat128 has quit IRC20:27
pabelangerjeblair: Ya, it would be nice to have the feedback20:27
pabelangerjeblair: clarkb: ^ think that is the most basic trigger we could have20:27
*** mat128 has joined #openstack-infra20:27
*** jrist has quit IRC20:28
*** mat128 has quit IRC20:29
openstackgerritOpenStack Proposal Bot proposed openstack/os-client-config master: Updated from global requirements  https://review.openstack.org/48808420:30
*** eharney has quit IRC20:30
*** jcoufal has quit IRC20:31
jeblairpabelanger: +2 but with a suggested revision20:32
*** ijw_ has joined #openstack-infra20:32
*** kgiusti has quit IRC20:33
pabelangerjeblair: sure!20:33
*** ijw_ has quit IRC20:33
*** ijw has joined #openstack-infra20:33
clarkbdhellmann: commented on https://review.openstack.org/#/c/488172/220:33
dhellmannclarkb : so just leave the AllowOverride lines alone and add the new line?20:34
*** ijw has quit IRC20:35
pabelangerjeblair: I'm unsure if status-url is needed, should I leave that empty for now?20:35
dhellmannclarkb : do I need to modify the docs.vhost file too?20:35
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add github connection to check pipeline  https://review.openstack.org/48818620:36
clarkbdhellmann: yes to allowoverride left alone and yes to modify docs vhost too20:36
dhellmannk20:36
openstackgerritDoug Hellmann proposed openstack-infra/system-config master: allow redirects in .htaccess files on the static web servers  https://review.openstack.org/48817220:36
dhellmannclarkb : ^^20:36
*** jtomasek has joined #openstack-infra20:36
dhellmannI had the impression from the apache docs that I needed AllowOverrides FileInfo in order for the AllowOverridesList directive to work, but maybe I misunderstood20:38
clarkbdhellmann: I dont think it is but maybr I misunderstood :)20:40
*** ijw has joined #openstack-infra20:40
dhellmannclarkb : I guess we'll see :-)20:40
*** jkilpatr has joined #openstack-infra20:40
dhellmannI'll place my bet on you understanding better20:40
*** xyang1 has joined #openstack-infra20:41
jeblairpabelanger: yes, leave success and failure just the way you wrote them20:41
pabelangerjeblair: great20:41
clarkbdhellmann: fungi probably knows20:42
dhellmannclarkb : maybe we can pull him away from those sausages for a few minutes20:43
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Start storing results with the SQL reporter  https://review.openstack.org/48795020:44
mordredpabelanger, jeblair: ok ^^ maybe third times the charm today (pipeline definitions are not normally high-merge-conflict bait)20:45
jeblairclarkb, dhellmann: my understanding was that we can use none with list (so i agree with clark and how the patch is currently written)20:47
jeblairclarkb, dhellmann: does anyone know why fileinfo is there?20:47
dhellmanncool, works for me20:47
dhellmannI do not20:47
openstackgerritMerged openstack-infra/puppet-zuul master: Add PyMySQL python library for SQL Reporter support  https://review.openstack.org/48797420:47
openstackgerritMerged openstack-infra/system-config master: Enable SQL support for zuul v3 scheduler  https://review.openstack.org/48797520:47
openstackgerritMerged openstack-infra/system-config master: Update docs with correct information about github apps  https://review.openstack.org/48816720:47
openstackgerritMerged openstack-infra/system-config master: Remove ssh key from github connection and add to executor  https://review.openstack.org/48817820:47
jeblairdhellmann: oh, that's for docs.o.o isn't it?20:48
dhellmannjeblair : it is. Maybe just because we have a .htaccess file in the root dir?20:49
*** jtomasek has quit IRC20:49
jeblairdhellmann: i'm guessing, yeah, it was added to "make .htaccess" work.  fileinfo includes redirect and redirectmatch, so that's actually a noop change for docs.o.o.20:49
dhellmannoh, so the list directive doesn't limit the other directive?20:50
dhellmannit's additive?20:50
*** rlandy has quit IRC20:50
jeblairif that's the only htaccess file we have now, and those are the only directives in it, now might be a good time to further restrict it.20:50
dhellmannsure, let's do that20:50
*** trown is now known as trown|outtypewww20:51
dhellmannI was just checking our htaccess, and those are the only directives I see20:51
jeblairdhellmann: yes, my understanding is it's a union.20:51
jeblairthough, to be fair, i don't see a specific config example to clarify that :)  (the only examples i see are from different sets)20:52
openstackgerritDoug Hellmann proposed openstack-infra/system-config master: allow redirects in .htaccess files on the static web servers  https://review.openstack.org/48817220:52
dhellmannhow's that one? ^^20:52
*** priteau has quit IRC20:53
dhellmannjeblair , clarkb : I have to drop offline soon. I'll check for more comments on that patch (or here) first thing in the morning. Thanks for your help!20:54
*** yland has joined #openstack-infra20:54
*** jrist has joined #openstack-infra20:55
pabelangerclarkb: jeblair: mind a +3 on https://review.openstack.org/488186 ?21:07
*** rhallisey has quit IRC21:10
mordredand https://review.openstack.org/#/c/48795021:10
fungiclarkb: dhellmann: sorry to disappoint, but the best i can offer is to pull up the apache config docs and perform a dramatic reading21:12
*** hashar has quit IRC21:12
*** yamamoto_ has joined #openstack-infra21:13
*** yland has quit IRC21:14
fungiwhat apache know-how i have is mostly hard-won from trial and error followed by attempting to interpret the documentation based on observed behaviors21:15
*** yland has joined #openstack-infra21:15
*** ldnunes has quit IRC21:15
*** yland is now known as Guest8551421:15
*** Goneri has joined #openstack-infra21:18
*** baoli has quit IRC21:18
*** yamamoto_ has quit IRC21:20
*** thorst has quit IRC21:25
*** thorst has joined #openstack-infra21:26
*** thorst_ has joined #openstack-infra21:27
clarkbjeblair: I do not know why fileinfo is there21:28
*** andreww has quit IRC21:28
openstackgerritMerged openstack-infra/project-config master: Add github connection to check pipeline  https://review.openstack.org/48818621:29
clarkbI am getting dinner going so it is rwady tonight then can look at apache stuff more21:29
openstackgerritMerged openstack-infra/project-config master: Start storing results with the SQL reporter  https://review.openstack.org/48795021:30
*** thorst has quit IRC21:30
*** Douhet has quit IRC21:30
*** rbrndt has quit IRC21:31
*** esberglu has quit IRC21:31
jeblairclarkb, fungi, dhellmann: i left a comment on that change.  i think it's good with that caveat.  i did perform a find through all of afs/o.o/docs for htaccess files, and the root and infra/pub are the only ones.21:32
*** thorst_ has quit IRC21:32
*** Guest85514 is now known as yland21:33
pabelangerjeblair: mordred: looks like an issue with mysql config change for zuulv3: http://paste.openstack.org/show/616791/21:34
*** Goneri has quit IRC21:34
*** gongysh has joined #openstack-infra21:36
*** gongysh has quit IRC21:36
mordredjeblair: is the zuul running with the zuul.conf changes making sure the mysql connection is present?21:37
*** e0ne has joined #openstack-infra21:37
jeblairpabelanger: and scheduler was reconfigured or restarted after the mysql connection was added to config?21:37
mordredpabelanger: gha - you21:37
openstackgerritMatthew Treinish proposed openstack-infra/project-config master: Add support for post-processing stestr runs to run_tox.sh  https://review.openstack.org/48821221:37
mordredyah. that21:37
mtreinishclarkb, fungi: ^^^ there's no rush on that, it's just a future proofing step21:38
mordredjeblair: the scheduler does the config validation right, not the executor?21:38
*** rbrndt has joined #openstack-infra21:38
jeblairmordred: correct21:39
mordredjeblair: cool. I just had a thought of "do we need to list mysql connection in theexecutor config so that it knows ... but then realized not"21:39
*** Douhet has joined #openstack-infra21:40
*** jamesdenton has quit IRC21:40
jeblairmtreinish, pabelanger, mordred: we were actually just talking about needing to soft-freeze the jenkins scripts in prep for the zuulv3 transition.  so we should decide whether we want that in now, or later.  we actually have already done most of the tox stuff, so if we want that in now, we'll need to go update it.  but it's not hard.21:40
*** jamesdenton has joined #openstack-infra21:42
mtreinishjeblair: well I've just been doing a bunch of stestr work the past week to try and get it in a better place to switch things over to it21:42
mordredmaybe mtreinish would find porting that patch to v3 fun?21:42
mtreinishI was hoping to have something running with it in the gate by the ptg21:42
mtreinishmordred: it's just ansible right21:42
mordredmtreinish: yup21:42
mtreinishpersonally I prefer bash, but it shouldn't be too hard to do in ansible21:42
mordredmtreinish: pabelanger is just finishing up the main tox role - we can ping you when he's done21:42
*** Apoorva has quit IRC21:43
*** Apoorva has joined #openstack-infra21:44
mtreinishmordred: heh, the patch is just if -d .testrepository use 'testr last', if -d .stestr use 'stestr last'21:44
jeblairmtreinish: cool, i think it's fine to get in now.  note that as we get closer to the ptg, the freeze on that area will need to get harder (so if you do need to make changes, you may need to update both simultaneously)21:44
jeblairmtreinish: but that's so simple, i'm guessing that won't be much of a burden :)21:44
*** esberglu has joined #openstack-infra21:45
mordredyah - we mostly want to make sure we don't, you know, undo work you've done when we do the cutover21:45
mordredsince I'm pretty sure that would be unpleasant21:45
*** aeng has joined #openstack-infra21:45
*** thorst has joined #openstack-infra21:45
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Collect logging information into ara callback  https://review.openstack.org/48785321:48
*** gordc has quit IRC21:48
pabelangerjeblair: mordred: yes, scheduled was reconfigured21:49
*** esberglu has quit IRC21:49
*** thorst has quit IRC21:50
pabelangermtreinish: ya, won't be hard to add that into zuul-jobs. can help tomorrow21:50
clarkbdhellmann: jeblair oh I meant to keep the existing fileinfo allow as is for backward compat21:50
clarkbbut maybe forcetype to the list is a better solution to that problem21:50
mordredpabelanger: reconfigured but not restarted?21:50
pabelangermordred: right21:50
*** esberglu has joined #openstack-infra21:51
mordredpabelanger: k. so we might need a full restart for this one21:51
mordredjeblair: ^^21:52
pabelangerack21:52
jeblairclarkb: yeah, dhellmann and i had a conversation about that.  his research indicated that /.htaccess didn't need anything other than those two.  i then found out after he left that *we* have htaccess files that use forcetype, but i'm still okay with the change to see if we need it.  (ours is just copypasta)21:52
*** jascott1 has quit IRC21:53
*** jascott1 has joined #openstack-infra21:53
clarkbjeblair: ah so we likely don't even need it for publications?21:53
jeblairmordred, pabelanger: well, if either of you want to look into that, that would be great (my list is growing faster than shrinking at the moment).  but i guess if you just want to restart it, you can do that and tell me you're "collecting more data".  :)21:53
*** armax has joined #openstack-infra21:53
jeblairclarkb: right.  eg: ./infra/publications/puppet-overview/help/.htaccess21:53
pabelangerjeblair: sure, let me restart and collect more data21:54
clarkbjeblair: I can go ahead and approve it as is and we can see I guess. everyone ready for that? I think dhellmann has popped out but I don't think he needs to be around for this since we don't quite have htaccess files yet21:54
jeblairclarkb: wfm21:54
clarkbdone21:54
*** jascott1 has quit IRC21:55
*** jascott1 has joined #openstack-infra21:55
pabelangerjeblair: mordred: restarted zuulv3.o.o, looks better21:58
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Collect logging information into ara callback  https://review.openstack.org/48785322:00
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Remove callback_whitelist setting  https://review.openstack.org/48821422:00
mordredpabelanger: woot!22:00
mordredpabelanger: I can verify that there are tables in the db22:02
pabelangermordred: nice22:02
mordredmysql> select * from alembic_version \G22:02
mordred*************************** 1. row ***************************22:02
mordredversion_num: f86c9871ee6722:02
mordredso now we just need a build to, you know, do a thing - I just pushed up two new patch revisions so we should have it soon enough22:03
openstackgerritPaul Belanger proposed openstack-infra/project-config master: WIP: testing  https://review.openstack.org/48821522:03
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref  https://review.openstack.org/48821622:03
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref  https://review.openstack.org/48821622:04
*** yamamoto has joined #openstack-infra22:04
pabelangermordred: woot22:05
pabelanger2017-07-27 22:04:00,044 DEBUG zuul.ProjectParser: Applying template gtest-org/ansible to pipeline check22:05
openstackgerritMerged openstack-infra/system-config master: allow redirects in .htaccess files on the static web servers  https://review.openstack.org/48817222:05
pabelangermordred: must still have something not configured properly, not seeing events from github22:06
pabelangerwonder if we need to update logging.conf22:07
*** lbragstad has quit IRC22:08
*** xarses has joined #openstack-infra22:08
*** e0ne has quit IRC22:09
openstackgerritMerged openstack-infra/shade master: Updated from global requirements  https://review.openstack.org/48594322:10
*** gouthamr_ has joined #openstack-infra22:10
*** gouthamr has quit IRC22:10
*** thorst has joined #openstack-infra22:13
*** thorst has quit IRC22:14
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Monitor job root and kill over limit jobs  https://review.openstack.org/48590222:15
pabelangermordred: how can I tell if openstack-zuul is listening on gtest-org/ansible?22:15
clarkbpabelanger: for github? I am not sure you can since its an async ping from github to zuul22:16
*** armax has quit IRC22:17
pabelangerclarkb: ya, trying to figure out if the event stream is setup22:17
clarkbwell there is no event stream22:17
clarkbaiui what happens is when github wants to send you an event it does an http request against your server22:17
pabelangeroh22:18
*** armax has joined #openstack-infra22:18
pabelangerdo we need to open firewall?22:18
clarkbthat is a possibility22:18
pabelangerhmm22:18
clarkbpabelanger: did you configure github to do the callback?22:18
pabelangerreading docs now22:18
clarkbyou also need to make sure github knows where to make the request22:18
pabelangerclarkb: no? I mean, we added the github app 'openstack-zuul'22:18
jeblairstill need more i think22:19
pabelangerya22:19
pabelangerI think we need to open ports22:19
pabelangerfor github webhook22:19
jeblairpabelanger: zuulv3 already has port 80 open22:19
pabelangerk22:20
jeblairpabelanger: https://docs.openstack.org/infra/zuul/feature/zuulv3/admin/drivers/github.html#github22:20
pabelangerjeblair: yes, reading that now22:20
pabelangerhttp://zuulv3.openstack.org/connection/github/payload22:20
pabelangeradding that to webhook now22:21
jeblairi have no idea why this is required if you install an app :|22:21
pabelangernot sure too22:21
jeblairSpamapS: ^ is that still accurate?22:21
clarkbisn't a github app just a registration of a system user type thing?22:22
clarkbI guess its a little more than that but you'd still have to configure it to tell it where to talk to22:23
pabelanger2017-07-27 22:23:08,278 DEBUG zuul.GithubWebhookListener: Github Webhook Received.22:23
pabelangercool22:23
mordredpabelanger: wait- no22:23
pabelangerokay22:23
pabelangerdisabling22:23
mordredwe should absolutely not need to configure a webhook on an individual repo22:23
pabelangermordred: this was under gtest-org/settings/hooks22:24
mordredright.22:24
mordredthe app was already added to the org22:24
pabelangerdoes the app have our webhook settings?22:24
mordredwebhook is how you configure things that are not apps22:24
pabelangerk22:25
pabelangerI am unsure how we get the events from github currently22:25
mordredpabelanger: https://github.com/organizations/openstack-infra/settings/apps/openstack-zuul22:25
mordredhas https://zuulv3.openstack.org/connection/github/payload configured as webhook URL22:25
pabelangerOh22:25
pabelangerya22:25
pabelangerwe don't have https22:26
clarkbdo you have to apply the app to the repo?22:26
clarkbah https vs http22:26
*** rcernin has quit IRC22:26
mordredah - gotit22:26
mordredI fixed22:26
pabelangerack22:26
mordredyay!22:26
mordredso - in any case - that page: https://github.com/organizations/openstack-infra/settings/apps/openstack-zuul is where the app is configured22:26
mordred(I also added to docs today - but it's all new and whatnot, so making sure we all know where the things are is important)22:27
pabelangermordred: is there a test web hook button?22:27
clarkbre http, there is a webhook secret22:28
pabelangerwonder if User authorization callback URL should also be http22:28
*** skelso has quit IRC22:28
clarkbare we worried about that being disclosed? it could be used by something other than github to spoof github22:28
*** bobh has quit IRC22:28
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Update SQL reporter to store results  https://review.openstack.org/48822122:28
pabelangerI don't think we have a secret today, do we?22:28
clarkbpabelanger: that config page has something *'d out, but that could jus be a default22:28
*** iyamahat has joined #openstack-infra22:29
clarkbwe may also not validate it in zuul yet22:29
mordredwe do have a secret22:29
mordredand I'm not sure about it being exposed - we may want to consider adding https just to make sure22:29
*** rbrndt has quit IRC22:29
clarkbmordred: reading the docs what it would allow someone else to do is pretend to be github and submit jobs. But that only matters if we are validating the secret in zuul too22:30
clarkb(not sure if we validate)22:30
pabelangerOh22:30
pabelangerUnder advance22:30
pabelangerI see some logs22:30
pabelangerhttps://github.com/organizations/openstack-infra/settings/apps/openstack-zuul/advanced22:30
pabelangerOH SNAP22:31
pabelangerhttp://zuulv3.openstack.org/22:31
mordredpabelanger: woot22:32
pabelangerI clicked redeliver button22:32
pabelangerand re-sent the event22:32
jeblairclarkb: https://review.openstack.org/473674  suggests we're doing something with webook token secrets22:32
*** ihrachys has quit IRC22:33
jeblairpabelanger: i don't see anything about zuul having started22:33
*** ihrachys has joined #openstack-infra22:33
clarkbjeblair: it sure does and that looks liek what I would expect based on reading github docs22:33
pabelangerjeblair: same, not sure why that is. Looking at logs currently22:34
openstackgerritMonty Taylor proposed openstack-infra/zuul feature/zuulv3: Update SQL reporter to store results  https://review.openstack.org/48822122:34
jeblairyeah, i think we should start httpsing22:34
*** makowals has quit IRC22:34
mordredjeblair: ++22:34
pabelangeragree22:34
*** rbrndt has joined #openstack-infra22:34
mordredBUT - I'm going to EOD for today ... exciting stuff today errbody22:35
jeblairyep22:35
jeblairand i think we can proceed with this for now and just change our secret when we https)22:35
*** rbrndt has quit IRC22:35
pabelangerYa, very exciting22:35
pabelangerjeblair: seems like a good idea22:35
*** gouthamr_ has quit IRC22:37
jeblairclarkb, pabelanger: oh this doc update from mordred clarifies things: https://review.openstack.org/48816522:37
jeblair(and makes me wonder whether the app approach may be okay over http -- i dunno)22:38
*** iyamahat_ has joined #openstack-infra22:38
*** andreww has joined #openstack-infra22:38
*** rama_y_ has joined #openstack-infra22:38
*** openstack has joined #openstack-infra22:43
*** sflanigan has joined #openstack-infra22:43
*** sflanigan has joined #openstack-infra22:43
*** jlvillal has joined #openstack-infra22:44
*** med_ has quit IRC22:44
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Update docs on github connection settings  https://review.openstack.org/48816522:44
*** med_ has joined #openstack-infra22:44
*** baoli has joined #openstack-infra22:44
*** stevebaker has quit IRC22:44
*** dangers_` has joined #openstack-infra22:44
*** med_ is now known as Guest1393622:45
*** stevebaker has joined #openstack-infra22:45
*** ggillies_ has quit IRC22:45
*** ggillies_ has joined #openstack-infra22:45
*** sflanigan has quit IRC22:45
*** sflanigan has joined #openstack-infra22:45
*** sflanigan has joined #openstack-infra22:45
*** dangers_away has quit IRC22:45
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Stop double-logging the build id  https://review.openstack.org/48818322:45
*** kiltzman has quit IRC22:45
*** oomichi has quit IRC22:45
*** numans has quit IRC22:45
*** samP has quit IRC22:45
*** flepied has joined #openstack-infra22:45
pabelangermordred: Is it possible that app_key should be a filename, that holds the key?22:45
jeblairpabelanger: hrm, it sure looks like it has the right access.  i wonder if there's a bug with the app id/key stuff.22:46
*** mriedem is now known as mriedem_afk22:46
jeblairpabelanger: using the dev version of github3?22:46
*** ihrachys has joined #openstack-infra22:46
*** kiltzman has joined #openstack-infra22:46
pabelangerjeblair: I think this is the issue: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/driver/github/githubconnection.py?h=feature/zuulv3#n40922:46
pabelangerjeblair: yes, installed from github22:47
*** lbragstad has joined #openstack-infra22:47
pabelangerhttps://github.com/BonnyCI/hoist/blob/59975a64ea077ec01a6fbb69847e29011b51443c/inventory/group_vars/all#L3522:47
jeblairpabelanger: do we have a "Failed to open" line in our logs?22:47
*** oomichi has joined #openstack-infra22:47
pabelangerjeblair: surprisingly no22:48
pabelangerbonncyci is using a file it seems22:48
*** pleia2 has joined #openstack-infra22:48
*** jascott1 has quit IRC22:48
mordredthe code in zuul has an app_key or app_key_file22:48
mordredbut maybe the thing I put into the app_key is not the right data ...22:48
*** numans has joined #openstack-infra22:48
*** jascott1 has joined #openstack-infra22:48
pabelangerI can test quickly if using a file for app_key works22:49
mordredwell - we can also just generate a new app_key real quick so we're sure we've got the right data ...22:49
*** jrist has quit IRC22:49
pabelangersure, that works22:49
jeblairmordred: no there's no app_key_file option22:49
jeblairmordred: app_key *is* the file22:49
*** baoli has quit IRC22:49
mordredjeblair: oh - really? ok. I totally misread that then22:50
jeblairmordred: i think we should make the code be like what's in your brain22:50
*** bradm has joined #openstack-infra22:50
mordredwell - lemme make a new secret real quick so that we're sure what we've got is the rightthing ...22:50
mordredit's possible what I put there is just wrong and the data makes no sense to not be in a file22:51
*** jascott1 has quit IRC22:51
jeblairpabelanger: and yes, we're missing a bunch of github logs22:51
*** jascott1 has joined #openstack-infra22:51
*** xinliang has quit IRC22:51
clarkbwhy wouldn't appkey go in the zuul.conf for the github connection?22:51
pabelangerjeblair: possible we don't have correct logging.conf settings now?22:52
mordredyes. I have DEFINITELY not put the right data in22:52
jeblairpabelanger: the right solution is to change the logging names in the github driver.22:52
clarkbis it some massive base64 string?22:52
pabelangerjeblair: ack22:52
jeblairpabelanger: you could fix it with logging.conf, but let's not.  let's fix the code.  it should match gerrit.22:52
pabelanger++22:52
jeblairgotta run22:52
mordredclarkb: yes! it's a pem-encoded rsa key22:52
mordredthe thing I had in there before was the oauth secret22:53
pabelangerah, that explains why it is a file22:53
jeblair++22:53
*** gouthamr has joined #openstack-infra22:53
*** annegentle has quit IRC22:53
clarkbmordred: ah22:53
mordredyup. I'm going to put it in hiera now22:53
jeblairso we may not need to change anything about the config syntax, just fix logging.22:54
pabelangerprogress is being made! Exciting22:54
mordredok. I put the content in zuul_github_app_key in heira22:55
mordredobviously we'll need to update puppet-zuul and site.pp and write the file somewhere22:55
pabelangerk, I can do that tomorrow morning22:55
clarkbEmilienM: just as a heads up I'm seeing much smaller tripleo job log dirs now (under 100MB). I am still seeing a ton of redundant and unnecesasry /etc data still being copied though. Can we get those cleaned up and then call it good? (then work on moving to whitelist of files over time?)22:55
pabelangermordred: did you manually write it to disk on zuulv3.o.o in the mean time?22:56
mordredcool. I also removed the app_key setting from private hiera - we can add the filename to public hiera when we update site.pp22:56
mordredpabelanger: I did not22:56
pabelangerack22:56
*** jascott1 has quit IRC22:58
*** jascott1 has joined #openstack-infra22:59
*** Apoorva_ has joined #openstack-infra22:59
*** xyang1 has quit IRC23:00
*** yland has quit IRC23:00
*** jascott1_ has joined #openstack-infra23:01
*** jascott1 has quit IRC23:01
*** yland has joined #openstack-infra23:01
SpamapSjeblair: reading it now23:01
*** yland is now known as Guest2248723:01
yamamotoif zuul "jobs" section has multiple matching entries for a job and they have different "branch" specified, which one will be used?23:02
*** Apoorva has quit IRC23:02
yamamotothe last one?23:02
*** jascott1_ has quit IRC23:02
clarkbyamamoto: yes last match wins23:02
yamamotothanks23:02
*** Apoorva_ has quit IRC23:02
*** jascott1 has joined #openstack-infra23:02
*** hongbin has quit IRC23:03
*** Apoorva has joined #openstack-infra23:03
*** lrossetti has joined #openstack-infra23:04
*** xinliang has joined #openstack-infra23:04
*** xinliang has quit IRC23:04
*** xinliang has joined #openstack-infra23:04
*** Douhet has quit IRC23:04
SpamapSjeblair: ah yeah, that is entirely accurate AFAIK.23:05
SpamapSalso something to consider.. if you don't https, perhaps worse than people spoofing github is people spoofing zuul and thus convincing other projects that they can merge evil code.23:06
*** jascott1 has quit IRC23:07
*** Douhet has joined #openstack-infra23:07
*** samP has joined #openstack-infra23:07
*** Guest22487 has quit IRC23:11
*** lbragstad has quit IRC23:12
clarkbSpamapS: the connection back to github is via https though23:14
clarkbSpamapS: its only the webhook that fires over http aiui23:15
*** thorst has joined #openstack-infra23:15
SpamapSsame problem.23:16
clarkbhow?23:16
SpamapSIf somebody MITM's zuul, github will be pushing events to them.23:16
SpamapSand then they can do whatever they want with those events23:16
clarkbyes but those events aren't useful in the way you described23:17
SpamapSeven not even talk to our zuul at all23:17
clarkbyou'd have to mitm github to convince zuul to merge things23:17
clarkbthere are two channels of communication. First is github -> zuul for webhooks that trigger jobs.23:17
clarkbthe second is zuul -> github to post results/merge stuff23:17
*** Douhet has quit IRC23:17
clarkbonly the first is potentially compromised in this setup23:17
SpamapSRight, so what I can do with that is first create a PR with evil code..23:18
SpamapSthen wait for an event for GOOD code23:18
SpamapSmanipulate that event to test and merge my EVIL code23:18
*** Douhet has joined #openstack-infra23:18
SpamapSNow, this is all plainly visible in git history23:18
SpamapSso it's not an earth shattering vulnerability23:19
SpamapSbut it's still going to merge the evil PR23:19
SpamapSbecause my MITM said "hey they approved PR#3733723:19
*** ijw has quit IRC23:19
clarkbwhen code is fetched I thought the pr path was used23:19
fungiis there any metadata we can double-check? like with gerrit we get an event which could _potentially_ trigger a job, then zuul queries gerrit for details like approvals before proceeding23:19
clarkbso you couldn't spoof that23:20
SpamapSclarkb: not sure I follow.23:20
SpamapSfungi: entirely possible we do that too23:20
*** thorst has quit IRC23:21
*** sdague has quit IRC23:21
clarkbSpamapS: I may be assuming too muh about the data integrity of the github api23:21
clarkbSpamapS: but you get PRs and sha1s right? if those don't line up I'd expect something to go derp?23:21
clarkbor are you saying you'd just write a completely bogus "foo is approved" event?23:21
SpamapSclarkb: If they both came from the event though.. MITM'd event can make them match.23:22
clarkbso you wouldn't even need to wait for anything23:22
fungiit's not like spoofing gerrit events, if it were even plausible to do so, would have much actual impact on zuul v2 right now i don't think23:22
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref  https://review.openstack.org/48821623:22
clarkbbut ya thats why I pointed out earlier that it doesn't matter right now because zuul isn't merging anything23:22
SpamapSclarkb: I'm reading the github driver now to see if it works the way I think23:22
SpamapSEntirely out of curiosity.23:22
SpamapSI think it's a no-brainer to HTTP that webhook.23:22
clarkb*HTTPS and definitely23:23
clarkbI was assessing the impact of not doing so immediately23:23
clarkbwhich I think is fairly minimal from a problem perspective. People can push bad copde and we test it23:23
clarkbor people send bogus events and we test bad code23:23
clarkb(in both cases the risk of DOS and running bad code exists whether or not we http or https. The difference is whether or not it is ever recorded in github)23:24
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Remove v2-full-legacy experimental job  https://review.openstack.org/48822923:24
*** jrist has joined #openstack-infra23:24
clarkbI suppose we could also be used to spam comments on pull requests which would be unfortunate23:25
fungilbyl should fix that, right?23:25
SpamapSHTTPS yes, derp23:26
clarkbfungi: lbyl?23:26
fungilook before you leap23:26
SpamapSclarkb: from what I see, the event dictates the PR# and details. Other things are fetched from the API, but ultimateyl, they're fetched from that PR.23:26
fungiquery to determine the veracity of the event23:27
clarkbfungi: right we do that with gerrit to check mergeability status but not sure if github does23:27
SpamapSpull_request_review is the event that usually triggers an approval23:27
clarkbSpamapS: so I think ^ is what it comes down to? if github driver checks PR for proper approvals before merging then that isn't a problem23:27
fungido the pr details indicate what repo they're proposed against?23:27
clarkbfungi: yes they are indexed per repo so they have to23:28
clarkbgerrit uses global change number but github uses PR specific number23:28
clarkber project specific PR numbers23:28
SpamapSI don't see that event handler doing any checks23:28
*** Guest22487 has joined #openstack-infra23:28
SpamapShttps://github.com/openstack-infra/zuul/blob/feature/zuulv3/zuul/driver/github/githubconnection.py#L210-L22523:29
SpamapSbody is the submitted event body23:29
clarkbSpamapS: right so we'd test it and say +2 that worked, but before submitting (at least in with gerrit) zuul checks if the change meets the criteria23:30
clarkbif github driver did the same thing I think you'd avoid that particular problem23:30
SpamapSoh wow23:30
SpamapSso you can just say "mike approved PR#EVIL"23:30
SpamapSis that secret used to sign the details?23:31
SpamapSthat would be nice23:31
clarkbSpamapS: right but if zuul then hit github (via https) to check the PR details then just saying mike approved foo would not be sufficient23:31
SpamapSI think it is actually used to sign the details.23:31
SpamapSsha1 tho23:32
clarkbSpamapS: yes the secret is used to sign the details (it hashes them and you both derive the same hash to check validity)23:32
clarkbSpamapS: there is a method to get the pull request reviews from github at least23:33
SpamapSDefinitely23:33
clarkbI don't yet see if it is being checked as the source of truth rather than the PR content23:33
SpamapSI'm sure we get a direct ID for the review that we can just query.23:33
SpamapSBut...23:33
SpamapSNot sure this matters if the secret signs.23:33
SpamapSMITM can't sign w/o the secret.23:33
clarkbcanMerge just returns True :(23:33
SpamapSclarkb: u can haz merge!23:34
SpamapSalways23:34
SpamapS:-/23:34
clarkbbut canMerge is where you'd implement that sort of checking23:34
clarkbto be extra cautious23:34
SpamapSso if the MITM can't create their own fictitious event from a real GIthub one, I don't think we have to worry23:35
SpamapSexcept maybe replay attacks23:35
*** jaypipes has quit IRC23:35
clarkboh that was the concern secret is optional and we weren't sure if we had a secret but mordred said we did23:35
clarkbso maybe the thing we should be sure to document is "USE A SECRET"23:35
*** dingyichen has joined #openstack-infra23:35
SpamapSYeah I'm surprised it is optional :-/23:35
clarkband also implement canMerge based on change.reviews23:35
SpamapS    def _validate_signature(self, request):23:35
SpamapS        secret = self.connection.connection_config.get('webhook_token', None)23:35
SpamapS        if secret is None:23:36
SpamapS            return True23:36
clarkbya23:36
SpamapSmaybe a warning. :)23:36
clarkbor just make it required23:36
SpamapSI like the idea of 'sploding without one. There's zero reason not to have one.23:36
clarkbgithub may not require it but zuul should23:36
clarkb++23:36
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Fix branches for v2 jobs  https://review.openstack.org/48823123:36
* SpamapS makes patch23:36
SpamapSGithub's take: https://developer.github.com/webhooks/securing/23:38
SpamapSso non-chalant23:38
clarkbat least they recommend a constant time string equality comparison23:38
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Fix branches for v2 jobs  https://review.openstack.org/48823123:39
clarkbI don't know that zuul's implementation does that23:39
*** Apoorva_ has joined #openstack-infra23:40
SpamapSI think it was just left optional so tests didn't have to set one.23:41
clarkbstreql is an apache2 licensed implementation that will do constant time str comparisons. Or we can easily write our own that does not short circuit or do fancy prefix magic on comparison23:41
clarkbbasically iterate over each character and check regardless of any already known state23:42
*** Apoorva has quit IRC23:43
SpamapShm23:44
SpamapSclarkb: seems like we should add a story for that one. :)23:45
SpamapSyeah so making it required breaks the tests.. need to start submitting with a signature23:45
SpamapSbut not too hard to go ahead and do that since we already have an implementation23:46
*** thorst has joined #openstack-infra23:46
clarkbSpamapS: do you want me to work on a patch to do constant time comparison?23:47
SpamapSclarkb: almost done and you'll need to stack on me.23:49
clarkbok23:49
SpamapSclarkb: (almost done making it required, not constant time)23:49
*** baoli has joined #openstack-infra23:50
*** thorst has quit IRC23:55
*** thorst has joined #openstack-infra23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!