Tuesday, 2015-10-06

*** geoffarnold has quit IRC00:00
*** dingyichen has joined #openstack-infra00:00
lifelessclarkb: python -c 'import sys; print(sys.path)'00:00
*** yamamoto has joined #openstack-infra00:01
clarkblifeless: hrm /usr/local is after00:02
clarkbshouldn't those be reversed?00:02
lifelessclarkb: only if you want it to be useful00:02
lifelessorder should be stdlib, local, distro00:02
clarkblifeless: ya stdlib is first then a dynload thing then local then distro00:04
*** ddieterl_ has quit IRC00:05
lifelessclarkb: then it should have won, no ? gnrrg00:05
*** lykinsbd has joined #openstack-infra00:09
*** tjones has quit IRC00:10
*** wenlock_ has quit IRC00:10
*** lykinsbd has quit IRC00:11
*** lykinsbd has joined #openstack-infra00:11
*** ajmiller has quit IRC00:12
*** wenlock1 has quit IRC00:12
*** otter768 has quit IRC00:13
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: (Re)add midonet tempest jobs  https://review.openstack.org/23093200:14
*** otter768 has joined #openstack-infra00:14
*** zz_dimtruck is now known as dimtruck00:16
*** gildub has joined #openstack-infra00:16
*** tsekiyama has quit IRC00:17
*** weshay has joined #openstack-infra00:18
*** tsekiyama has joined #openstack-infra00:18
openstackgerritMerged openstack/requirements: Use oslosphinx for doc build  https://review.openstack.org/23094400:19
*** geoffarn_ has quit IRC00:20
*** geoffarnold has joined #openstack-infra00:21
openstackgerritMerged openstack/requirements: Updated oslo.cache to 0.7.0  https://review.openstack.org/22785000:21
*** achanda has joined #openstack-infra00:21
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Use overlay always to make SERVICE_HOST setup easy  https://review.openstack.org/22997000:25
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Update query for bug 1501366  https://review.openstack.org/23126200:26
openstackbug 1501366 in OpenStack Compute (nova) "libvirtError: Error while building firewall: Some rules could not be created for interface" [High,Confirmed] https://launchpad.net/bugs/150136600:26
openstackgerritMerged openstack/requirements: Add weakmethod backport library (for 2.7 and 2.6 only)  https://review.openstack.org/22591100:26
clarkbsc68cal: I think latest version of 229970 should work with multinode based on the errors00:26
clarkbsc68cal: but that assumes there aren't more errors :)00:26
*** hdd has quit IRC00:27
*** sflanigan has joined #openstack-infra00:27
*** sflanigan has joined #openstack-infra00:27
*** pvaneck has quit IRC00:28
*** mtanino has quit IRC00:29
*** tiswanso has joined #openstack-infra00:29
*** julim has joined #openstack-infra00:30
openstackgerritJoshua Harlow proposed openstack-dev/pbr: Escape some of the changelog RST messages  https://review.openstack.org/22756600:30
openstackgerritMerged openstack-infra/irc-meetings: Telco Working Group no longer alternates  https://review.openstack.org/23099900:31
clarkbjust ogt an email from rax saying identity endpoint IPs are changing00:31
clarkbthis may require a nodepool restart at some point00:32
clarkb(I can't remember how deeply that stuff gets cached)00:32
*** glauco has joined #openstack-infra00:32
*** camunoz has quit IRC00:32
jeblairclarkb: i hope and expect not; only relevant caching would be request pools or something.00:33
*** tiswanso has quit IRC00:33
openstackgerritMerged openstack-infra/elastic-recheck: Update query for bug 1501366  https://review.openstack.org/23126200:34
openstackbug 1501366 in OpenStack Compute (nova) "libvirtError: Error while building firewall: Some rules could not be created for interface" [High,Confirmed] https://launchpad.net/bugs/150136600:34
openstackgerritMerged openstack-infra/irc-meetings: Ended alternating schedule for Containers Team  https://review.openstack.org/23102300:35
openstackgerritJoshua Harlow proposed openstack-dev/pbr: Ensure changelog complete is written even if it already exists  https://review.openstack.org/23126800:35
*** vmbrasseur is now known as vmb_out00:36
*** tsekiyama has quit IRC00:36
*** glauco has quit IRC00:37
*** ayoung has quit IRC00:37
*** fedexo has joined #openstack-infra00:39
*** julim has quit IRC00:41
*** davideagnello has quit IRC00:42
*** geoffarn_ has joined #openstack-infra00:42
*** davideagnello has joined #openstack-infra00:44
*** bharathm has quit IRC00:45
*** geoffarnold has quit IRC00:46
*** gildub has quit IRC00:46
*** davideagnello has quit IRC00:46
*** davideagnello has joined #openstack-infra00:47
*** Sukhdev has quit IRC00:47
*** glauco_ has joined #openstack-infra00:49
*** camunoz has joined #openstack-infra00:49
*** sridhar_ram has joined #openstack-infra00:49
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Add documentation for third-party-ci setup  https://review.openstack.org/22758400:50
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Create a single-node 3rd party ci system  https://review.openstack.org/20033000:50
*** dims_ has joined #openstack-infra00:50
*** sridhar_ram has quit IRC00:54
*** sridhar_ram has joined #openstack-infra00:54
*** dimtruck is now known as zz_dimtruck00:54
*** zz_dimtruck is now known as dimtruck00:55
*** dims_ has quit IRC00:55
*** gyee has quit IRC00:57
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Create a single-node 3rd party ci system  https://review.openstack.org/20033000:57
*** dims_ has joined #openstack-infra00:58
*** geoffarn_ has quit IRC01:03
*** geoffarnold has joined #openstack-infra01:03
*** bpokorny has joined #openstack-infra01:04
*** ayoung has joined #openstack-infra01:05
*** baoli has joined #openstack-infra01:06
openstackgerritRamy Asselin proposed openstack-infra/nodepool: Update nodepool to use secure.conf file  https://review.openstack.org/18976201:07
*** alexsyip has quit IRC01:08
*** bharathm has joined #openstack-infra01:08
*** yamamoto has quit IRC01:08
*** Swami has joined #openstack-infra01:10
*** bpokorny has quit IRC01:10
*** fedexo has quit IRC01:11
*** alivigni has joined #openstack-infra01:12
*** ybathia has quit IRC01:12
*** lykinsbd has quit IRC01:13
*** ivar-lazzaro has quit IRC01:15
*** ivar-lazzaro has joined #openstack-infra01:16
*** tqtran_ has quit IRC01:16
openstackgerritAri proposed openstack-infra/jenkins-job-builder: Implements: additions for OS3 plugin entry points in builders and scm  https://review.openstack.org/22910001:17
*** kengo_sakai has joined #openstack-infra01:18
*** baoli has quit IRC01:19
*** baoli has joined #openstack-infra01:20
*** dimtruck is now known as zz_dimtruck01:21
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Add a db api method to get a list of test_runs by test.test_id  https://review.openstack.org/23128201:21
*** geoffarnold has quit IRC01:24
*** geoffarnold has joined #openstack-infra01:24
*** asselin_ has joined #openstack-infra01:27
*** alivigni has quit IRC01:27
*** markvoelker has quit IRC01:28
*** Swami has quit IRC01:28
*** dims_ has quit IRC01:30
*** hdd has joined #openstack-infra01:31
*** asselin_ has quit IRC01:31
*** Daisy has joined #openstack-infra01:33
*** baoli has quit IRC01:34
*** mriedem_away has quit IRC01:35
*** tsekiyama has joined #openstack-infra01:37
*** baoli has joined #openstack-infra01:37
*** sridhar_ram1 has joined #openstack-infra01:38
*** Daisy has quit IRC01:38
*** rguillebert has quit IRC01:38
*** Daisy has joined #openstack-infra01:39
*** sridhar_ram1 has quit IRC01:39
*** sridhar_ram has quit IRC01:41
*** btully has quit IRC01:41
*** tsekiyama has quit IRC01:42
*** Daisy has quit IRC01:42
*** Daisy has joined #openstack-infra01:42
*** glauco_ has quit IRC01:43
*** glauco has joined #openstack-infra01:45
*** geoffarn_ has joined #openstack-infra01:46
*** geoffarnold has quit IRC01:46
*** baoli has quit IRC01:48
*** sekrit has joined #openstack-infra01:48
*** baoli has joined #openstack-infra01:48
*** lykinsbd has joined #openstack-infra01:49
clarkbsc68cal: the nova net job for multinode worked but neutron not so happy. I am guessing that neutron deletes those iptables rules or something01:53
clarkbwill try to pick it up tomorrow01:53
*** stevemar_ has quit IRC01:59
*** ddieterly has joined #openstack-infra02:00
*** stevemar_ has joined #openstack-infra02:00
*** rfolco has joined #openstack-infra02:02
*** geoffarn_ has quit IRC02:07
*** geoffarnold has joined #openstack-infra02:07
*** tiswanso has joined #openstack-infra02:07
*** odgrim has quit IRC02:07
*** baoli has quit IRC02:08
*** yamamoto has joined #openstack-infra02:09
*** tiswanso_ has joined #openstack-infra02:10
*** yamamoto has quit IRC02:13
*** tiswanso has quit IRC02:13
*** rfolco has quit IRC02:14
*** camunoz has quit IRC02:15
*** yamahata has quit IRC02:15
*** unicell has quit IRC02:18
*** rkukura has quit IRC02:19
*** bharathm has quit IRC02:20
*** bharathm has joined #openstack-infra02:22
*** wenlock has joined #openstack-infra02:24
*** tlian2 has quit IRC02:25
*** weshay has quit IRC02:25
*** gildub has joined #openstack-infra02:25
*** geoffarnold has quit IRC02:26
*** ivar-lazzaro has quit IRC02:26
*** bharathm has quit IRC02:26
*** odgrim has joined #openstack-infra02:27
*** Daisy has quit IRC02:28
*** geoffarnold has joined #openstack-infra02:28
*** sridhar_ram has joined #openstack-infra02:31
*** odgrim has quit IRC02:31
*** stevemar_ has quit IRC02:41
*** stevemar_ has joined #openstack-infra02:42
*** stevemar_ has quit IRC02:43
*** stevemar_ has joined #openstack-infra02:43
*** glauco has quit IRC02:45
*** glauco_ has joined #openstack-infra02:45
*** sdake has joined #openstack-infra02:49
*** achanda has quit IRC02:50
*** jamielennox has quit IRC02:51
*** sridhar_ram has quit IRC02:54
*** btully has joined #openstack-infra02:55
*** ayoung has quit IRC02:58
*** btully has quit IRC03:00
openstackgerritIan Wienand proposed openstack/diskimage-builder: dib-lint: ignore blank lines in element ordering  https://review.openstack.org/23129203:02
*** spzala has quit IRC03:04
*** markvoelker_ has joined #openstack-infra03:05
*** sdake has quit IRC03:09
*** hdd has quit IRC03:09
*** sdake has joined #openstack-infra03:10
*** kushal has joined #openstack-infra03:10
*** geoffarn_ has joined #openstack-infra03:11
*** geoffarnold has quit IRC03:12
*** jamielennox has joined #openstack-infra03:13
*** sdake has quit IRC03:13
*** harlowja_at_home has joined #openstack-infra03:15
*** hdd has joined #openstack-infra03:15
*** yamamoto has joined #openstack-infra03:17
*** camunoz has joined #openstack-infra03:17
*** bpokorny has joined #openstack-infra03:18
*** bpokorny has quit IRC03:18
*** bpokorny has joined #openstack-infra03:19
*** angdraug has quit IRC03:19
*** stevemar_ has quit IRC03:21
*** markvoelker has joined #openstack-infra03:22
*** stevemar_ has joined #openstack-infra03:22
*** markvoelker_ has quit IRC03:25
*** markvoelker_ has joined #openstack-infra03:26
*** markvoelker has quit IRC03:28
openstackgerritIan Wienand proposed openstack-infra/project-config: diskimage-builder element cleanups for dib-lint  https://review.openstack.org/22973003:30
*** sdake has joined #openstack-infra03:33
*** tiswanso_ has quit IRC03:37
*** odgrim has joined #openstack-infra03:38
*** tjones has joined #openstack-infra03:41
*** ddieterly has quit IRC03:42
*** DericHorn-HP has joined #openstack-infra03:47
*** links has joined #openstack-infra03:48
openstackgerritMerged openstack-infra/project-config: Add experimental devstack f22 job  https://review.openstack.org/23028803:49
*** DericHorn-HP has quit IRC03:51
*** geoffarn_ has quit IRC03:53
*** geoffarnold has joined #openstack-infra03:53
*** DericHorn-HP has joined #openstack-infra03:57
*** glauco_ has quit IRC04:00
ianwLast reconfigured: Tue Oct 06 2015 09:02:57 GMT+1100 (AEDT) <- it is now 15:02 AEDT ... has zuul really not reloaded configs that long?04:02
*** shashank_hegde has quit IRC04:03
*** puranamr has joined #openstack-infra04:07
*** Sree has joined #openstack-infra04:07
*** Sree has joined #openstack-infra04:08
*** odgrim has quit IRC04:11
*** puranamr has quit IRC04:12
*** geoffarnold has quit IRC04:14
*** tjones has quit IRC04:15
*** harlowja_at_home has quit IRC04:15
*** geoffarnold has joined #openstack-infra04:15
*** Daisy has joined #openstack-infra04:17
*** bharathm has joined #openstack-infra04:23
*** Sukhdev has joined #openstack-infra04:23
*** bharathm has quit IRC04:27
openstackgerritIan Wienand proposed openstack-infra/project-config: Add f22 node matching  https://review.openstack.org/23129804:27
*** Daisy has quit IRC04:28
*** Daisy has joined #openstack-infra04:29
openstackgerritSteve Kowalik proposed openstack-infra/puppet-zanata: Move to Zanata server 3.7.2  https://review.openstack.org/23028304:31
*** Daisy has quit IRC04:33
*** geoffarn_ has joined #openstack-infra04:36
*** amotoki has joined #openstack-infra04:38
*** btully has joined #openstack-infra04:39
*** geoffarnold has quit IRC04:40
*** tjones has joined #openstack-infra04:41
*** achanda has joined #openstack-infra04:42
*** unicell has joined #openstack-infra04:42
*** ddieterly has joined #openstack-infra04:43
*** dims_ has joined #openstack-infra04:46
*** ddieterly has quit IRC04:47
*** nijaba has quit IRC04:51
*** skylerberg_ has quit IRC04:52
*** DericHorn-HP has quit IRC04:54
*** DericHorn-HP has joined #openstack-infra04:56
*** geoffarn_ has quit IRC04:57
*** geoffarnold has joined #openstack-infra04:57
*** abregman has joined #openstack-infra05:00
*** nijaba has joined #openstack-infra05:01
*** asettle has quit IRC05:01
*** asettle has joined #openstack-infra05:01
*** _fortis has quit IRC05:02
*** schang has quit IRC05:02
*** cbader_ has joined #openstack-infra05:02
*** charz has quit IRC05:02
*** filler has quit IRC05:02
*** jpeeler has quit IRC05:02
*** morgan has quit IRC05:02
*** jesusaurus has quit IRC05:02
*** cebruns has quit IRC05:03
*** redrobot has quit IRC05:03
*** gothicmindfood has quit IRC05:03
*** coreycb has quit IRC05:03
*** jaredrohe has quit IRC05:03
*** cbader has quit IRC05:03
*** cp16net has quit IRC05:03
*** sdague has quit IRC05:03
*** cebruns has joined #openstack-infra05:04
*** lyz has quit IRC05:04
*** tristanC has quit IRC05:04
*** gothicmindfood has joined #openstack-infra05:05
*** lyz_ has joined #openstack-infra05:05
*** tristanC has joined #openstack-infra05:05
*** cp16net has joined #openstack-infra05:05
*** _fortis has joined #openstack-infra05:06
*** morgan has joined #openstack-infra05:06
*** asettle has quit IRC05:06
*** asettle has joined #openstack-infra05:07
*** jpeeler has joined #openstack-infra05:07
*** coreycb has joined #openstack-infra05:07
*** schang has joined #openstack-infra05:08
*** sdague has joined #openstack-infra05:08
*** redrobot has joined #openstack-infra05:08
*** redrobot is now known as Guest2867805:09
*** f1ller has joined #openstack-infra05:09
*** charz has joined #openstack-infra05:09
*** jaredrohe has joined #openstack-infra05:09
*** sdake_ has joined #openstack-infra05:09
*** geoffarnold has quit IRC05:11
*** jesusaurus has joined #openstack-infra05:11
*** sdake has quit IRC05:13
*** rkukura has joined #openstack-infra05:15
*** bpokorny has quit IRC05:15
*** armax has quit IRC05:16
*** mrmartin has joined #openstack-infra05:17
*** geoffarnold has joined #openstack-infra05:19
*** asmigala has joined #openstack-infra05:19
*** BharatK has joined #openstack-infra05:21
*** rlandy has joined #openstack-infra05:25
*** yamahata has joined #openstack-infra05:27
*** davideagnello has quit IRC05:30
*** davideagnello has joined #openstack-infra05:30
*** Sree_ has joined #openstack-infra05:31
*** Sree_ is now known as Guest1345805:31
*** asettle has quit IRC05:32
*** asettle has joined #openstack-infra05:32
*** Sree has quit IRC05:34
*** sdake_ has quit IRC05:38
*** jswarren has quit IRC05:41
*** bharathm has joined #openstack-infra05:42
*** jswarren has joined #openstack-infra05:42
*** ddieterly has joined #openstack-infra05:44
*** kushal has quit IRC05:46
*** hdd has quit IRC05:46
*** DericHorn-HP has quit IRC05:47
*** ddieterly has quit IRC05:48
*** hdd has joined #openstack-infra05:52
*** unicell has quit IRC05:52
*** unicell has joined #openstack-infra05:53
*** unicell1 has joined #openstack-infra05:59
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders no longer use nodedb for diskimages  https://review.openstack.org/22239805:59
*** unicell has quit IRC06:00
*** ildikov has quit IRC06:00
*** geoffarnold has quit IRC06:00
*** ildikov has joined #openstack-infra06:00
*** geoffarnold has joined #openstack-infra06:01
*** AJaeger has joined #openstack-infra06:06
*** MarkAtwood has joined #openstack-infra06:06
*** cloudtrainme has joined #openstack-infra06:08
*** rkukura has quit IRC06:09
openstackgerritAndreas Jaeger proposed openstack/requirements: Update constraints for openstackdocstheme 1.2.3  https://review.openstack.org/23132506:09
*** tjones has quit IRC06:10
*** asettle has quit IRC06:11
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/23132706:12
odyssey4mehow do I become authorized to change a channel topic with chanserv?06:12
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/23132806:12
*** scheuran has joined #openstack-infra06:12
*** hichihara has quit IRC06:15
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image delete using gearman  https://review.openstack.org/19811406:16
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders no longer use nodedb for diskimages  https://review.openstack.org/22239806:16
*** oomichi has joined #openstack-infra06:18
*** MarkAtwood has quit IRC06:18
*** e0ne has joined #openstack-infra06:22
*** markvoelker_ has quit IRC06:24
*** k4n0 has joined #openstack-infra06:24
*** ParsectiX has joined #openstack-infra06:27
*** mrmartin has quit IRC06:29
*** hdd has quit IRC06:29
*** e0ne has quit IRC06:30
*** oomichi has quit IRC06:30
*** Sukhdev has quit IRC06:33
*** uberjay has quit IRC06:33
*** markvoelker has joined #openstack-infra06:35
*** Guest13458 has quit IRC06:36
*** uberjay has joined #openstack-infra06:36
*** Sree has joined #openstack-infra06:37
*** stevemar_ has quit IRC06:38
*** vlaza has joined #openstack-infra06:39
*** markvoelker has quit IRC06:40
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/23132806:41
*** jamielennox is now known as jamielennox|away06:41
*** wenlock has quit IRC06:42
*** geoffarnold has quit IRC06:43
*** geoffarnold has joined #openstack-infra06:43
*** ddieterly has joined #openstack-infra06:45
*** markvoelker has joined #openstack-infra06:45
*** ifarkas has joined #openstack-infra06:46
yolandagood morning06:46
*** johnsom_ has quit IRC06:46
*** johnsom_ has joined #openstack-infra06:47
*** gildub has quit IRC06:48
*** woodster_ has quit IRC06:49
*** markvoelker has quit IRC06:50
yolandajhesketh, care revisiting https://review.openstack.org/#/c/228604/ ?06:50
*** ddieterly has quit IRC06:50
*** devvesa has joined #openstack-infra06:52
*** matrohon has joined #openstack-infra06:56
*** ekarlso has quit IRC06:58
*** markvoelker has joined #openstack-infra07:00
*** sabeen has quit IRC07:02
*** peristeri has joined #openstack-infra07:03
*** geoffarnold has quit IRC07:04
*** markvoelker has quit IRC07:04
*** geoffarnold has joined #openstack-infra07:04
*** jlanoux has joined #openstack-infra07:05
*** nadya has joined #openstack-infra07:05
openstackgerritMerged openstack-infra/project-config: Make new functional jobs voting  https://review.openstack.org/22948807:06
*** mrmartin has joined #openstack-infra07:08
*** cloudtrainme has quit IRC07:08
*** pahuang has quit IRC07:09
yolandajhesketh, also... are we ok to land https://review.openstack.org/#/c/202867/ ? as it's zuul, i prefer to check with you07:09
*** matbu_ has quit IRC07:09
*** matbu has joined #openstack-infra07:10
*** pahuang has joined #openstack-infra07:12
*** sergmelikyan has joined #openstack-infra07:13
*** markvoelker has joined #openstack-infra07:14
*** btully has quit IRC07:16
*** sshnaidm has joined #openstack-infra07:16
*** jcoufal has joined #openstack-infra07:17
*** achanda has quit IRC07:18
*** jcoufal has quit IRC07:19
*** markvoelker has quit IRC07:19
*** jcoufal has joined #openstack-infra07:19
*** ildikov has quit IRC07:21
*** achanda has joined #openstack-infra07:22
*** sergmelikyan has quit IRC07:23
*** matbu has quit IRC07:23
*** tmorin has joined #openstack-infra07:23
*** ekarlso has joined #openstack-infra07:24
openstackgerritColleen Murphy proposed openstack-infra/glean: Allow node hostname to be null  https://review.openstack.org/23134107:25
*** flepied1 has quit IRC07:25
*** matbu has joined #openstack-infra07:26
*** MarkAtwood has joined #openstack-infra07:26
openstackgerritMerged openstack-infra/puppet-httpd: Allow specification of locations to not be proxied  https://review.openstack.org/23002907:27
*** mpavone has joined #openstack-infra07:28
*** markvoelker has joined #openstack-infra07:29
*** megm_ has quit IRC07:32
*** arxcruz has joined #openstack-infra07:32
*** markvoelker has quit IRC07:33
*** sergmelikyan has joined #openstack-infra07:33
mrmartinyolanda: I just saw your comment about the git module.07:38
mrmartindo you have any idea which module is using that now?07:38
*** megm has joined #openstack-infra07:38
yolandanibalizer, you should know more about the nanliu-git, you added that to modules.env right?07:39
nibalizerso nanliu-git was renamed puppetcommunity-git and then puppetcommunity-git_resource07:39
mrmartinI do a quick search on system-config, but found nothing yet. Anyway I think we can freely upgrade to puppetcommunity-git07:39
yolandaah, then it's the same module07:39
*** matbu_ has joined #openstack-infra07:39
mrmartinnibalizer: still here? I quickly fix the pull request.07:40
nibalizerif you fix it i can merge it07:40
mrmartinjust the puppetcommunity-git is getting better and better now.07:40
*** matbu has quit IRC07:40
nibalizerwell git_resource07:40
yolandaok, then just that rename is needed07:40
nibalizerya on system-config07:40
*** notnownikki has joined #openstack-infra07:41
yolandaso mrmartin, if you can file a change for that rename, on a separate patch, i'll review, and then you can add your change on top07:42
*** markvoelker has joined #openstack-infra07:43
openstackgerrityolanda.robla proposed openstack-infra/puppet-cgit: Add symlink on local git directory  https://review.openstack.org/23045907:44
*** hashar has joined #openstack-infra07:45
mrmartinnibalizer: we are waiting for the Travis CI run, but the updated code is there: https://github.com/puppet-community/puppet-git_resource/pull/1707:46
*** ddieterly has joined #openstack-infra07:46
*** geoffarn_ has joined #openstack-infra07:46
mrmartinyolanda: let's wait the new release of puppet-community git resource, and I'll make the patch.07:47
yolandaok, do you know when it will land?07:48
*** markvoelker has quit IRC07:48
mrmartinyeah, when nibalizer merge the code.07:48
*** geoffarnold has quit IRC07:50
*** yamahata has quit IRC07:50
*** ddieterly has quit IRC07:51
*** HeOS has quit IRC07:51
*** MarkAtwood has quit IRC07:53
*** sergmelikyan has quit IRC07:53
*** ParsectiX has quit IRC07:54
mrmartinawesome, can you create a 0.3.0 release?07:56
*** MarkAtwood has joined #openstack-infra07:56
*** achanda has quit IRC07:57
*** rossella_s has quit IRC07:58
*** markvoelker has joined #openstack-infra07:58
*** ParsectiX has joined #openstack-infra07:58
nibalizerif you write me a changelog07:59
openstackgerritThomas Herve proposed openstack/requirements: Add websocket-client to requirements  https://review.openstack.org/22305907:59
nibalizerif you make a PR that updates the changelog and metadata.json and stuff I will merge it and cut a 0.3.0 release07:59
mrmartinI do it.08:00
*** rossella_s has joined #openstack-infra08:00
*** mpaolino has joined #openstack-infra08:01
*** MarkAtwood has quit IRC08:02
*** rguillebert has joined #openstack-infra08:02
*** markvoelker has quit IRC08:02
*** achanda has joined #openstack-infra08:05
*** mrmartin has quit IRC08:06
*** geoffarn_ has quit IRC08:07
*** geoffarnold has joined #openstack-infra08:08
*** flepied has joined #openstack-infra08:08
*** btully has joined #openstack-infra08:08
*** Ala has joined #openstack-infra08:10
*** ildikov has joined #openstack-infra08:11
*** mrmartin has joined #openstack-infra08:11
*** markvoelker has joined #openstack-infra08:13
*** btully has quit IRC08:13
*** kmartin has quit IRC08:13
*** kmartin has joined #openstack-infra08:13
*** achanda has quit IRC08:16
*** johnsom__ has joined #openstack-infra08:17
*** markvoelker has quit IRC08:18
*** johnsom_ has quit IRC08:20
*** eantyshev has joined #openstack-infra08:22
*** MarkAtwood has joined #openstack-infra08:23
*** jistr has joined #openstack-infra08:25
*** claudiub has joined #openstack-infra08:27
*** markvoelker has joined #openstack-infra08:27
*** Sree_ has joined #openstack-infra08:27
*** mrmartin has quit IRC08:28
*** Sree_ is now known as Guest6684008:28
*** geoffarnold has quit IRC08:29
*** geoffarnold has joined #openstack-infra08:29
*** vmb_out has quit IRC08:30
*** patrickeast has quit IRC08:30
*** vmb has joined #openstack-infra08:30
*** mjb has quit IRC08:30
*** mjb has joined #openstack-infra08:30
*** Sree has quit IRC08:31
*** warpc has quit IRC08:31
*** mrmartin has joined #openstack-infra08:32
*** markvoelker has quit IRC08:32
*** patrickeast has joined #openstack-infra08:32
*** warpc has joined #openstack-infra08:34
*** dtantsur|afk is now known as dtantsur08:34
mrmartinnibalizer: the git release pull request is here: https://github.com/puppet-community/puppet-git_resource/pull/1908:34
*** dizquierdo has joined #openstack-infra08:35
*** dizquierdo_ has joined #openstack-infra08:35
*** sergmelikyan has joined #openstack-infra08:37
*** achanda has joined #openstack-infra08:39
*** HeOS has joined #openstack-infra08:40
*** markvoelker has joined #openstack-infra08:42
*** dingyichen has quit IRC08:45
mrmartinnibalizer: thanks!08:45
*** ddieterly has joined #openstack-infra08:46
*** markvoelker has quit IRC08:47
*** dizquierdo_ has quit IRC08:48
*** dizquierdo_ has joined #openstack-infra08:49
*** mrmartin has quit IRC08:49
*** geoffarnold has quit IRC08:50
*** achanda has quit IRC08:50
*** geoffarnold has joined #openstack-infra08:50
*** ddieterly has quit IRC08:50
*** flepied has quit IRC08:52
*** bharathm has quit IRC08:55
*** markvoelker has joined #openstack-infra08:56
*** ZZelle has quit IRC09:01
*** markvoelker has quit IRC09:01
*** ZZelle has joined #openstack-infra09:01
*** pelix has joined #openstack-infra09:03
openstackgerritDmitry Tantsur proposed openstack/requirements: Bump ironic-lib to the latest release for Mitaka  https://review.openstack.org/23136809:04
*** markvoelker has joined #openstack-infra09:06
*** MarkAtwood has quit IRC09:07
openstackgerritThomas Morin proposed openstack-infra/project-config: networking-bgpvpn: add devstack job  https://review.openstack.org/23035809:09
*** flepied has joined #openstack-infra09:10
*** markvoelker has quit IRC09:10
*** geoffarnold has quit IRC09:11
*** geoffarnold has joined #openstack-infra09:12
*** markvoelker has joined #openstack-infra09:13
*** markvoelker has quit IRC09:18
*** flepied has left #openstack-infra09:20
*** flepied has joined #openstack-infra09:20
*** sergmelikyan has quit IRC09:21
*** markvoelker has joined #openstack-infra09:22
wznoinskfungi: on the other hand I wouldn't mind useing something like cgit web frontend to get the age of the repos and sync depending on that, it doesn't seem to give you any api or at least script friendly interface09:24
*** markvoelker has quit IRC09:26
*** mrmartin has joined #openstack-infra09:32
*** markvoelker has joined #openstack-infra09:32
*** geoffarnold has quit IRC09:33
*** geoffarn_ has joined #openstack-infra09:33
*** jlanoux has quit IRC09:34
*** jtomasek has joined #openstack-infra09:35
*** MarkAtwood has joined #openstack-infra09:35
*** markvoelker has quit IRC09:39
openstackgerritMarton Kiss proposed openstack-infra/system-config: Update nanliu's git module to puppet-community git  https://review.openstack.org/23139109:40
*** deepakcs has joined #openstack-infra09:42
*** gildub has joined #openstack-infra09:46
*** markvoelker has joined #openstack-infra09:46
*** ddieterly has joined #openstack-infra09:47
*** markvoelker has quit IRC09:50
*** ddieterly has quit IRC09:52
*** MarkAtwood has quit IRC09:52
*** sergmelikyan has joined #openstack-infra09:52
*** pblaho has quit IRC09:53
*** geoffarn_ has quit IRC09:54
*** geoffarnold has joined #openstack-infra09:54
*** dougshelley66 has quit IRC09:55
sambettsAJaeger, yolanda: Can I get some eyes on https://review.openstack.org/#/c/229060 please?09:57
dguitarbiteAJaeger: hello, can you add me to training-labs core please?09:57
*** _skolekonov is now known as skolekonov09:57
yolandahi sambetts looking09:57
sambettsyolanda: Thanks!09:59
yolandadguitarbite, i added you10:00
dguitarbiteyolanda: thanks10:00
*** markvoelker has joined #openstack-infra10:01
*** dougshelley66 has joined #openstack-infra10:02
*** e0ne has joined #openstack-infra10:03
mrmartinyolanda: will we brake anything with #231391 ?10:05
*** markvoelker has quit IRC10:05
mrmartinI not found any mark of this new git module in system-config, but we can have a dependent puppet module which is using it.10:06
*** rossella_s has quit IRC10:06
*** rossella_s has joined #openstack-infra10:07
yolandamrmartin, i don't think there is any module that started to use it10:07
*** Guest66840 has quit IRC10:08
*** Sree has joined #openstack-infra10:08
mrmartinI need to rebase https://review.openstack.org/#/c/231048/ too.10:09
*** jlanoux has joined #openstack-infra10:10
yolandaah yes, add on top10:10
*** markvoelker has joined #openstack-infra10:11
*** Sree has quit IRC10:13
*** jamielennox|away is now known as jamielennox10:14
*** kzaitsev1ws is now known as kzaitsev_ws10:15
*** geoffarnold has quit IRC10:15
*** markvoelker has quit IRC10:16
*** geoffarnold has joined #openstack-infra10:16
*** markvoelker has joined #openstack-infra10:17
openstackgerritMarton Kiss proposed openstack-infra/system-config: Fix git repository refresh trigger issue of askbot  https://review.openstack.org/23104810:17
*** mrmartin has quit IRC10:20
*** markvoelker has quit IRC10:22
openstackgerritTom Fifield proposed openstack-infra/groups-static-pages: Update Organiser Tips, new user-group faq page  https://review.openstack.org/23140010:25
*** markvoelker has joined #openstack-infra10:26
*** jlanoux has quit IRC10:27
*** mrmartin has joined #openstack-infra10:27
*** pc_m has joined #openstack-infra10:27
*** jlanoux has joined #openstack-infra10:27
*** mrmartin has quit IRC10:27
sambettsyolanda: Is there a way to make check-requirements non-voting for a specfic repo? or do I just have to disable it?10:30
yolandasambetts, you can add a voting:false flag on zuul layout.yaml10:31
yolandayou shall find some samples there you can filter by job name, branch, etc10:31
*** ociuhandu has quit IRC10:32
sambettsyolanda: Thanks!10:33
AJaegerdguitarbite: I can't - yolanda, can you add dguitarbite, please? see https://review.openstack.org/#/c/218345/10:33
yolandai did10:33
AJaegeryolanda: great, thanks!10:34
*** markvoelker has quit IRC10:34
* AJaeger should read more closely next time10:34
dguitarbiteAJaeger: She already added :)10:34
yolandano problem10:34
* dguitarbite scratches his head to fix CI issues10:34
AJaegersambetts: https://pypi.python.org/pypi/cisco-ironic-contrib/0.0.0 is setup but openstackci cannot upload any packages to it.10:36
*** Hal has joined #openstack-infra10:36
*** geoffarnold has quit IRC10:36
AJaegersambetts: do you really need the stable section?10:37
*** geoffarnold has joined #openstack-infra10:37
*** ihrachys has joined #openstack-infra10:37
*** jlanoux has quit IRC10:39
sambettsAJaeger: I based it off the networking-cisco config, do I not need it if we manage our own stable branches?10:39
*** Hal has quit IRC10:39
*** claudiub has quit IRC10:40
*** jlanoux has joined #openstack-infra10:40
sambettsAJaeger: also I've added openstackci to the ownership on pypi now10:40
AJaegersambetts: you only need it if you have another team manage stable branches, if you do it yourself, just remove it...10:40
*** Daisy has joined #openstack-infra10:41
sambettsAJaeger: Ok :) will do10:41
*** claudiub has joined #openstack-infra10:43
*** yamamoto has quit IRC10:43
*** ddieterly has joined #openstack-infra10:48
*** sdake has joined #openstack-infra10:52
*** ddieterly has quit IRC10:52
*** Daisy has quit IRC10:54
*** Daisy has joined #openstack-infra10:55
*** claudiub has quit IRC10:55
*** markvoelker has joined #openstack-infra10:55
*** trown|outttypeww is now known as trown10:56
*** ociuhandu has joined #openstack-infra10:57
*** baoli has joined #openstack-infra10:57
*** geoffarnold has quit IRC10:58
openstackgerritSam Betts proposed openstack-infra/project-config: Enable py3 jobs on networking-cisco  https://review.openstack.org/23141310:58
*** geoffarnold has joined #openstack-infra10:58
*** Daisy has quit IRC10:59
*** claudiub has joined #openstack-infra11:00
openstackgerritSam Betts proposed openstack-infra/project-config: Add cisco-ironic-contrib project  https://review.openstack.org/22906011:00
*** markvoelker has quit IRC11:01
*** ildikov has quit IRC11:02
*** ociuhandu has quit IRC11:02
*** ociuhandu_ has joined #openstack-infra11:02
*** amuller has joined #openstack-infra11:02
*** markvoelker has joined #openstack-infra11:05
*** BharatK has quit IRC11:06
*** baoli_ has joined #openstack-infra11:07
*** gildub has quit IRC11:08
*** baoli has quit IRC11:10
*** ddieterly has joined #openstack-infra11:10
*** markvoelker has quit IRC11:10
eantyshevHello, yolanda! May I ask you to review https://review.openstack.org/#/c/207094 , which you have already seen11:16
*** eantyshev has left #openstack-infra11:16
yolandahi eantyshev, sure, i'll look at it11:17
*** BharatK has joined #openstack-infra11:17
*** deepakcs has quit IRC11:19
*** markvoelker has joined #openstack-infra11:19
*** geoffarn_ has joined #openstack-infra11:20
*** geoffarnold has quit IRC11:23
*** amotoki has quit IRC11:24
*** markvoelker has quit IRC11:24
*** gildub has joined #openstack-infra11:24
*** dtantsur is now known as dtantsur|bbl11:24
*** nadya has quit IRC11:25
*** nikil89_ has joined #openstack-infra11:25
*** nikil89_ is now known as naggappan11:25
*** eantyshev has joined #openstack-infra11:26
*** nadya has joined #openstack-infra11:30
*** markvoelker has joined #openstack-infra11:34
*** ldnunes has joined #openstack-infra11:36
*** markvoelker has quit IRC11:38
*** gordc has joined #openstack-infra11:46
*** pblaho has joined #openstack-infra11:48
*** markvoelker has joined #openstack-infra11:48
*** rfolco has joined #openstack-infra11:52
*** k4n0 has quit IRC11:53
*** ociuhandu_ has quit IRC11:53
*** markvoelker has quit IRC11:53
*** ociuhandu has joined #openstack-infra11:54
*** jlanoux has quit IRC11:55
openstackgerritLenny Verkhovsky proposed openstack/os-testr: Adding support for combining more than 1 subunit file into single html Reason: We need to run some external scripts between tempest tests         and combime results into single html file.  https://review.openstack.org/23143911:56
*** weshay_xchat has joined #openstack-infra11:57
*** dprince has joined #openstack-infra11:57
*** jlanoux has joined #openstack-infra12:00
*** jlanoux has quit IRC12:01
*** jlanoux has joined #openstack-infra12:01
*** markvoelker has joined #openstack-infra12:03
*** mrmartin has joined #openstack-infra12:03
*** weshay_xchat is now known as weshay12:04
*** tiswanso has joined #openstack-infra12:05
*** tiswanso_ has joined #openstack-infra12:07
openstackgerritPaul Belanger proposed openstack-infra/project-config: Update row name for zuul pipelines  https://review.openstack.org/23144212:07
*** markvoelker has quit IRC12:07
*** bdemers has quit IRC12:07
AJaegergood morning, pabelanger12:09
*** sdake has quit IRC12:09
*** tiswanso has quit IRC12:10
*** sdake has joined #openstack-infra12:10
*** kgiusti has joined #openstack-infra12:11
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci: Use sudo for pacemaker status output  https://review.openstack.org/21153512:11
*** kengo_sakai has quit IRC12:13
*** markvoelker has joined #openstack-infra12:14
*** moravec has quit IRC12:17
*** markvoelker has quit IRC12:19
openstackgerritPaul Belanger proposed openstack-infra/project-config: Promote gate-project-config-grafyaml to gate pipeline  https://review.openstack.org/23144612:19
*** geoffarnold has joined #openstack-infra12:23
*** geoffarn_ has quit IRC12:24
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci: Use sudo for pacemaker status output  https://review.openstack.org/21153512:27
*** dra_ has joined #openstack-infra12:27
*** aysyd has joined #openstack-infra12:27
*** denisra has quit IRC12:27
*** burgerk has joined #openstack-infra12:27
*** nadya has quit IRC12:28
openstackgerritPaul Belanger proposed openstack-infra/project-config: Update row names for zuul status dashboard  https://review.openstack.org/23144212:28
*** ddieterly has quit IRC12:28
*** jcoufal has quit IRC12:28
*** markvoelker has joined #openstack-infra12:29
*** jcoufal has joined #openstack-infra12:29
*** MarkAtwood has joined #openstack-infra12:30
*** Daisy has joined #openstack-infra12:31
*** mrmartin has quit IRC12:32
*** amotoki has joined #openstack-infra12:35
*** shardy has joined #openstack-infra12:35
*** markvoelker has quit IRC12:36
*** ekarlso has quit IRC12:38
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci: Turn introspection back on  https://review.openstack.org/22593412:41
*** kushal has joined #openstack-infra12:42
*** markvoelker has joined #openstack-infra12:44
*** ttx has quit IRC12:44
*** ttx has joined #openstack-infra12:45
*** geoffarn_ has joined #openstack-infra12:45
*** geoffarnold has quit IRC12:45
*** annegentle has joined #openstack-infra12:45
*** edmondsw has joined #openstack-infra12:45
*** glauco_ has joined #openstack-infra12:46
*** asmigala has quit IRC12:46
*** mrmartin has joined #openstack-infra12:46
*** asmigala has joined #openstack-infra12:46
*** markvoelker has quit IRC12:48
*** bharathm has joined #openstack-infra12:49
*** sambetts is now known as sambetts-hungry12:50
*** pradk has joined #openstack-infra12:50
*** jaypipes has joined #openstack-infra12:50
*** julim has joined #openstack-infra12:50
*** denisra has joined #openstack-infra12:51
*** nadya has joined #openstack-infra12:53
*** ddieterly has joined #openstack-infra12:53
*** bharathm has quit IRC12:53
*** dra_ has quit IRC12:53
*** ekarlso has joined #openstack-infra12:56
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Test a update in trunk repository version  https://review.openstack.org/22978912:56
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source  https://review.openstack.org/21935112:57
*** denisra has quit IRC12:58
*** denisra has joined #openstack-infra12:58
*** markvoelker has joined #openstack-infra12:58
*** Daisy has quit IRC12:58
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source  https://review.openstack.org/21935112:58
*** markvoelker has quit IRC12:59
*** markvoelker has joined #openstack-infra12:59
*** spzala has joined #openstack-infra12:59
*** regXboi has joined #openstack-infra12:59
*** abregman has quit IRC12:59
*** markvoelker_ has joined #openstack-infra13:01
*** bswartz has joined #openstack-infra13:02
*** dkranz has joined #openstack-infra13:02
*** jhesketh has quit IRC13:03
*** markvoelker has quit IRC13:03
*** jhesketh has joined #openstack-infra13:04
*** dizquierdo has quit IRC13:05
*** dizquierdo_ has quit IRC13:05
*** geoffarn_ has quit IRC13:06
*** amotoki has quit IRC13:06
*** markvoelker has joined #openstack-infra13:06
openstackgerritIvan Kolodyazhny proposed openstack-infra/project-config: Increase timeout for cinder-dsvm-apache job  https://review.openstack.org/23146613:06
*** geoffarnold has joined #openstack-infra13:06
*** AJaeger has quit IRC13:08
*** sheeprine has quit IRC13:09
*** sheeprine has joined #openstack-infra13:09
*** denisra has quit IRC13:09
*** sdake has quit IRC13:09
*** ddieterl_ has joined #openstack-infra13:10
*** sdake has joined #openstack-infra13:10
*** markvoelker_ has quit IRC13:10
*** denisra has joined #openstack-infra13:12
openstackgerritTom Fifield proposed openstack-infra/groups-static-pages: Update Organiser Tips, new user-group faq page  https://review.openstack.org/23140013:12
*** sdake_ has joined #openstack-infra13:12
*** changbl has quit IRC13:13
*** maishsk has joined #openstack-infra13:13
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Replace oslo_log dependency with standard logging  https://review.openstack.org/23147013:13
*** ddieterly has quit IRC13:13
pabelangerfungi: ^ first patch removing oslo_log dependency from grafyaml13:14
*** sdake has quit IRC13:14
*** dra_ has joined #openstack-infra13:15
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Replace oslo_log dependency with standard logging  https://review.openstack.org/23147013:15
*** oomichi has joined #openstack-infra13:16
*** markvoelker has quit IRC13:17
*** moravec has joined #openstack-infra13:17
*** ildikov has joined #openstack-infra13:17
*** markvoelker has joined #openstack-infra13:17
*** denisra has quit IRC13:18
openstackgerritsean mooney proposed openstack-infra/project-config: Opening a repository for a ceilometer plugin for collectd.  https://review.openstack.org/22276313:19
*** alivigni has joined #openstack-infra13:20
*** gildub has quit IRC13:21
*** dkehn_ is now known as dkehn13:23
*** hichihara has joined #openstack-infra13:23
*** mriedem has joined #openstack-infra13:24
*** woodster_ has joined #openstack-infra13:25
*** thiagop has joined #openstack-infra13:27
openstackgerritZara proposed openstack-infra/storyboard-webclient: Resolve subscriptions faster- do one GET per list  https://review.openstack.org/22356213:27
*** geoffarnold has quit IRC13:27
*** geoffarnold has joined #openstack-infra13:27
openstackgerritPaul Michali proposed openstack-infra/project-config: Move VPN Functional tests to check queue  https://review.openstack.org/22304913:29
*** amotoki has joined #openstack-infra13:29
*** ParsectiX has quit IRC13:29
*** sambetts-hungry is now known as sambetts13:30
*** bhunter71 has quit IRC13:33
*** amotoki has quit IRC13:33
*** armax has joined #openstack-infra13:33
*** dra_ has quit IRC13:34
*** bdemers has joined #openstack-infra13:34
*** amotoki has joined #openstack-infra13:39
*** sabeen1 has joined #openstack-infra13:41
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova prep bdm rpc timeout failure  https://review.openstack.org/23148013:43
*** denisra has joined #openstack-infra13:45
*** rkukura has joined #openstack-infra13:45
*** ParsectiX has joined #openstack-infra13:46
*** burgerk has quit IRC13:46
*** oomichi has quit IRC13:48
*** Daisy has joined #openstack-infra13:48
*** mrmartin has quit IRC13:48
*** enikanorov has joined #openstack-infra13:48
openstackgerritgordon chung proposed openstack-infra/devstack-gate: [WIP] use ceilometer plugin  https://review.openstack.org/23148413:48
*** Sree has joined #openstack-infra13:49
*** amotoki has quit IRC13:49
*** mriedem has quit IRC13:49
*** mpavone has quit IRC13:52
*** radez` is now known as radez13:52
*** vmb is now known as vmbrasseur13:53
*** mrmartin has joined #openstack-infra13:55
*** MarkAtwood has quit IRC13:58
*** ParsectiX has quit IRC13:58
*** ParsectiX has joined #openstack-infra13:58
*** yamamoto has joined #openstack-infra13:59
Zarahi, storyboard here-- we're wondering whereabouts the planned maniphest migration is currently up to, since we haven't heard much (timezones may partly be to blame :)). We'd like to do a task tracker workroom at the summit (we plan to get some (up-to-date!) feedback beforehand and focus on a couple of key things).14:00
*** Sukhdev has joined #openstack-infra14:00
*** amotoki has joined #openstack-infra14:00
*** derekh has joined #openstack-infra14:01
*** asmigala has quit IRC14:01
Zarabut we want to be relevant to the rest of infra.14:02
*** dramalho has joined #openstack-infra14:03
*** ParsectiX has quit IRC14:04
*** mriedem has joined #openstack-infra14:04
*** mrmartin has quit IRC14:05
*** mpaolino has quit IRC14:05
*** rossella_s has quit IRC14:06
*** rossella_s has joined #openstack-infra14:07
clarkbZara: I think ruagair may be owrking on the db migration14:09
clarkbZara: which I believe is step 1 for making a dev deployment that we can iterate on14:09
clarkbsc68cal: for https://review.openstack.org/#/c/229970/ the neutron problems appear related to dhcp, VMs on the subnode are not getting leases. Nova net is not affected because dnsmasq runs on the subnode with multihost set to true14:10
*** sigmavirus24_awa is now known as sigmavirus2414:10
clarkbsc68cal: I think this is possibly related to the br-tun/br-int running over the existing gre tunnel which is for br-ex. Guessing we run into l2 filtering along the way14:10
Zaraclarkb: okay, thanks, I'll see if I can catch ruagair later :)14:11
openstackgerritLenny Verkhovsky proposed openstack/os-testr: Adding support for combining more than 1 subunit file into single html  https://review.openstack.org/23143914:11
clarkbsc68cal: ya on the old setup we didn't make an overlay for the non DVR case at all because everything is backhauled over br-int/br-tun to the network node14:13
clarkbI bet this does work in the dvr case, let me test that14:13
*** Guest28678 is now known as redrobot14:13
*** davideagnello has quit IRC14:14
*** davideagnello has joined #openstack-infra14:15
clarkbsc68cal: actually I wonder if neutron running on the compute host is confused by the existence of a br-ex at all? anyways I will likely have to hop on one of these test setups and see what networking looks likewhen a VM is booted14:15
clarkbbut I am beginning to think this idea of simplification may not simplify much in the neutron case and just make it more complicated14:16
*** burgerk has joined #openstack-infra14:16
*** burgerk_ has joined #openstack-infra14:16
*** eharney has joined #openstack-infra14:17
*** Daisy has quit IRC14:17
*** Daisy has joined #openstack-infra14:18
*** ayoung has joined #openstack-infra14:19
*** zz_dimtruck is now known as dimtruck14:19
*** Daisy has quit IRC14:19
*** Daisy has joined #openstack-infra14:20
yolandahi clarkb, question, do you clean up job history from jenkins periodically?14:20
clarkbyolanda: yes we have jenkins configured to retain only 24 hours of history14:20
*** burgerk has quit IRC14:20
yolandaok, we may miss that downstream, we hit some problems today for that14:20
clarkbyolanda: otherwise jenkins writes too many xml files to disk that it has to read every time you look at a job and it makes a denial of service against itself14:21
alivignipelix# I think I made the necessary changes you requested can you review again please when you get a chance14:21
yolandainodes problems14:21
yolandaclarkb, so it can cause problems in performance as well? didn't know14:21
clarkbyolanda: it also makes the jvm slow when it reads them all in to show you the colored balls14:21
yolandaclarkb, new thing learned then, i didn't know about it14:21
yolandaso do you have automated somehow?14:21
clarkbyolanda: its just a jenkins config somewhere14:22
clarkbyolanda: if you look in our defaults.yaml for jjb its the logrotate config there14:26
mriedemthis is holding up another change, has a +2 and been around for a few weeks, can i get another looker? https://review.openstack.org/#/c/224162/14:26
*** tonytan4ever has joined #openstack-infra14:27
*** wenlock has joined #openstack-infra14:27
anteayamriedem: I looked but you need a zuul core14:27
anteayawhich I am not14:27
openstackgerritJulia Kreger proposed openstack-infra/shade: WIP: node_set_provision_state wait/timeout support  https://review.openstack.org/21606214:27
yolandaah clarkb, cool, we'll need to adopt it14:27
*** stevemar_ has joined #openstack-infra14:28
*** cody-somerville has joined #openstack-infra14:28
*** fawadkhaliq has joined #openstack-infra14:28
*** dramalho has quit IRC14:28
*** achanda has joined #openstack-infra14:29
*** geoffarnold has quit IRC14:29
*** bswartz has quit IRC14:29
*** geoffarnold has joined #openstack-infra14:30
anteayayolanda: you could change your +1 to a +2 now on 224162 if you wanted14:31
*** jaypipes has quit IRC14:32
*** markvoelker_ has joined #openstack-infra14:32
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968514:33
dstufftanteaya: are the spikes better today on PyPI?14:33
anteayadstufft: that is a good question14:34
yolandaah anteaya, yes, thx for the reminder. I have several votes like that14:34
anteayammedvede: any feedback on your pypi experience in the last 48 hours?14:34
anteayayolanda: thank you14:34
*** dizquierdo has joined #openstack-infra14:34
eantyshevanteaya: Hello! I've collected 2 (+2)'s on https://review.openstack.org/207094 , what else needs to get it merged?14:35
anteayaeantyshev: hi, I am so sorry I missed the third party meeting today14:35
mmedvedeanteaya: it went good. took about 15 hours to pull down the 190Gb of packages though14:35
anteayaeantyshev: I don't know what happened I woke up and there was light outside my window14:35
mmedvedetook about 1 hour to setup server, so was easy14:35
*** jcoufal has quit IRC14:35
anteayaeantyshev: I don't know if my alarm failed or if I slept through it14:35
anteayaeantyshev: I'm sorry14:36
anteayammedvede: so that is feedback on your mirror setup, so thank you14:36
*** markvoelker has quit IRC14:36
anteayammedvede: dstufft is wanting feedback for fastly14:36
dstufftah so you've switched to a mirror now?14:36
anteayammedvede: have you any feedback on your experience hitting pypi.python.org in the last while?14:37
*** achanda has quit IRC14:37
eantyshevanteaya: I only wanted to 'promote' that zuul change of mine14:37
mmedvedeanteaya: oh, sorry. Yes, pypi stopped failing14:37
dstufftwe swtiched to a thing that will (hopefully) better route connections. yesterday (required a DNS change though, so propagation times)14:37
mmedvededstufft: not yet switched to a mirror.14:38
*** markvoelker_ has quit IRC14:38
dstufftmmedvede: ah great14:38
eantyshevanteaya: and very probably I was the only one who came for the meeting, no problem in that14:38
anteayadstufft: yay, thank you14:39
openstackgerritMerged openstack-infra/zuul: Add ZUUL_VOTING parameter  https://review.openstack.org/22416214:39
dstufftmmedvede: hopefully the new routing stuff is helping then14:39
anteayaeantyshev: well you were the only one who spoke up, but sometimes other folks don't say anything until others are talking14:39
anteayaeantyshev: I think since this is a change to the zuul scheduler that others might be waiting for jeblair or fungi or clarkb to approve14:39
anteayajust in case there are things that might be affected by this change and they are being prudent14:40
anteayadstufft: thank you so much for new pypi.python.org routing14:40
anteayadstufft: I'm grateful14:40
mmedvededstufft: yes. it looks good. Thank you for the fixes14:40
anteayadstufft: I'll keep an ear out for feedback and ping if I hear anything else14:41
dstufftanteaya: the thanks is mostly for Fastly :) I just whined at them and changed the DNS, I'll pass it along though :D14:41
eantyshevanteaya: I think it's time for me to ask one of them14:41
anteayadstufft: thank you for whining14:41
yolandafungi, anteaya, related to zuul there is https://review.openstack.org/#/c/202867/ , as it's zuul i wasn't confident to approve14:41
anteayadstufft: you are effective14:41
anteayaeantyshev: /me nods14:41
anteayayolanda: you are being prudent, and I commend that14:41
yolandayes, zuul is something critical14:42
*** cloudtrainme has joined #openstack-infra14:42
clarkbeantyshev: will that change allow invalid depends on to be ignored as well?14:42
*** MarkAtwood has joined #openstack-infra14:43
*** dtantsur|bbl is now known as dtantsur14:43
anteayayolanda: agreed14:43
*** wenlock_ has joined #openstack-infra14:43
*** dramalho has joined #openstack-infra14:43
*** tiswanso_ has quit IRC14:44
clarkbeantyshev: looks like it still queries gerrit so should not introduce new behavior there14:44
*** tlian has joined #openstack-infra14:44
*** tiswanso has joined #openstack-infra14:44
clarkbits purely making the zuul layout side handle the case where a dep doesn't exist in the layout14:44
eantyshevclarkb: yes, and what problem you see?14:45
clarkbeantyshev: no problem, just understanding the change14:45
*** Sukhdev has quit IRC14:45
*** MarkAtwood has quit IRC14:45
*** mrmartin has joined #openstack-infra14:46
eantyshevclarkb: BTW, what do you mean by invalid projects?14:46
clarkbeantyshev: invalid dependency, what happens if I provide a changeid that doesnt exist14:46
clarkb(I am not actually sure what happens in zuul currently but was curious if this changed that behavior)14:47
notnownikkisdague, hi, I noticed that bashate doesn't do a syntax check, and I've seen bash scripts that pass all style rules, but have invalid syntax. It seems like a simple check to do (just call bash -n), would it be accepted as a feature if I were to add it?14:47
*** xyang1 has joined #openstack-infra14:47
eantyshevclarkb: I didn't test exactly that, but I assume the change won't be enqueued if query to gerrit failed14:48
openstackgerritMerged openstack-infra/elastic-recheck: Add query for nova prep bdm rpc timeout failure  https://review.openstack.org/23148014:48
*** daemontool_ has quit IRC14:49
*** asselin_ has joined #openstack-infra14:49
*** odgrim has joined #openstack-infra14:50
*** maishsk_ has joined #openstack-infra14:50
clarkbeantyshev: ya I see the new code leaves the old valid change check in place14:50
*** geoffarnold has quit IRC14:50
clarkb(it runs after that exception would be raised)14:51
*** maishsk has quit IRC14:51
*** maishsk_ is now known as maishsk14:51
*** geoffarnold has joined #openstack-infra14:51
*** vlaza has quit IRC14:51
*** xyang1 has quit IRC14:53
*** tjones has joined #openstack-infra14:53
*** dizquierdo_ has joined #openstack-infra14:53
clarkbeantyshev: ok the one possible problem I see is this apperas to collapse the change queues14:54
*** dramalho has quit IRC14:54
clarkbeantyshev: which is fine for check but for gate would make everything serialize even if they don't have a shared set of tests?14:55
clarkbeantyshev: https://review.openstack.org/#/c/207094/9/zuul/scheduler.py line 180414:55
*** yamahata has joined #openstack-infra14:55
clarkboh thats only the independent manager, hrm. Not sure that we really want to have special behavior there14:55
*** markvoelker has joined #openstack-infra14:55
clarkbIf the project isn't in the layout we should handle that, but we shouldn't add it to the layout if that makes sense14:55
*** yamahata has quit IRC14:56
clarkbbecause this would still be broken in dependent pipeline managers14:56
*** hemnafk is now known as hemna14:57
eantyshevclarkb: how is that broken, could you expand on that14:57
*** achanda has joined #openstack-infra14:57
*** maishsk has quit IRC14:58
*** mrmartin has quit IRC14:58
clarkbeantyshev: zuul dynamically determines which projects are tested together based on shared tests in dependent pipeline managers14:58
*** bhunter71 has joined #openstack-infra14:58
clarkbeantyshev: based on that line above I think that zuul would not handle your use case in a dependent pipeline manager, it would only handle it for independent pipelines14:58
*** angdraug has joined #openstack-infra14:59
clarkbso my point is that zuul should handle it for both and I think the way to do that does not involve rewriting the config on the fly14:59
eantyshevclarkb: please, look at the testcase I made for dependent pipelines14:59
clarkb(this ignores whether or not this is actually a bug that should be fixed at all, one could argue that not providing the correct zuul refs on every test is a fail)15:00
clarkber not the refs I think the refs will be correct15:00
sc68calclarkb: it's possible that having a br-ex might be confusing neutron, based on the configuration15:00
*** derekh has quit IRC15:01
eantyshevclarkb: that test https://review.openstack.org/#/c/207094/9/tests/test_scheduler.py line 372215:01
*** arxcruz has quit IRC15:01
*** achanda has quit IRC15:01
sc68calclarkb: frankly I don't like having bridges that are named the same as bridges that neutron uses in the usual deployment case15:01
clarkbsc68cal: the problem is thats how neutron works :/15:02
clarkbsc68cal: so we could change the name sbut then neutron would stop working15:02
clarkbeantyshev: right so I think you want the independent pipeline to continue to not share a queue15:03
sc68calclarkb: neutron has configuration for all the bridge names15:03
clarkbsc68cal: but devstack does not15:03
clarkbsc68cal: so there is no way using devstack + neutron to say "use this interface"15:03
clarkbsc68cal: the exception for that is provider networks iirc15:03
clarkbeantyshev: so I guess the question is why don't you isolate the queue in independent pipelines as well?15:04
sc68calclarkb: we'd need to meet face to face I think and sort some of this stuff out15:04
*** Sree has quit IRC15:04
sc68calclarkb: mostly face 2 face so we can whiteboard15:05
clarkbsc68cal: not sure I agree, its a simple problem15:05
clarkbsc68cal: neutron + devstack requires br-ex to exist and if it does not you get a bridge unattached ot anything15:05
clarkbsc68cal: thats fine on single node because VMs don't need to tlak to anything15:05
*** davideagnello has quit IRC15:05
clarkbit breaks on multinode because you need VMs to talk to each other and tempest to talk to the VMs and dhcp and all those things15:06
clarkbthis means you have to preconfigure br-ex in a manner that includes that connectivity15:06
*** Ala has quit IRC15:06
*** hichihara has quit IRC15:06
*** david-ly_ is now known as david-lyle15:06
clarkbwhat we could do is make an overlay and call it something else. Then have a separate step to make br-ex15:06
sc68calclarkb: ok, but my continual frustration is that I am only getting information about the multinode case based on conversations with you15:07
clarkbsc68cal: its all in devstack and devstack-gate?15:07
sc68calclarkb: is it documented?15:07
sc68calI disagree15:07
clarkbhttps://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/multinode_diagram.txt is the overlay documentation15:08
eantyshevclarkb: dependent changes share the queue in both independent and dependent pipelines. For example, B (known) needs A(unkown). A is first enqueued and the queue is created. B enqueued to the tail of it15:08
clarkbeantyshev: thats not true for dependent pipelines, you can have depends on changes in other pipelines15:09
*** tlian2 has joined #openstack-infra15:09
sc68calclarkb: ok, well I guess I just disagree that a diagram is sufficient15:09
clarkbsc68cal: thats fair, but what is missing? no one has said "we need this additional info:15:09
clarkbsc68cal: is it more of the justification for the setup?15:09
clarkbor do we need more detail? both?15:09
eantyshevclarkb: other queues, perhaps?15:09
clarkbeantyshev: maybe thats my problem, confusing the terms15:10
clarkbeantyshev: let me rewrap my brain around it15:10
*** davideagnello has joined #openstack-infra15:10
sc68calclarkb: both detail and justification for setup would be helpful.15:10
anteayaclarkb: if I can get you an sc68cal together in a room for a period of time would you be adverse to that proposal?15:11
*** geoffarnold has quit IRC15:11
anteayaclarkb: my sense is that the way sc68cal consumes information is different than how you do15:11
clarkbsc68cal: for detail what are you looking for? (I am not sure what else should be added in addition to the interfaces, their connections, and IPs15:11
anteayaand I am feeling sc68cal would like to be supportive of a fix in this situation15:12
*** tlian has quit IRC15:12
clarkbanteaya: its possible, the problem is there is no reason to explicitly fly to some common place in the world to work through this15:12
clarkbits just not necessary and given my current situation at home I am less inclined to do that15:12
*** geoffarnold has joined #openstack-infra15:12
*** davideagnello has quit IRC15:12
anteayaclarkb: I understand15:12
clarkbI am happy to add more docs though, just trying to determine what specifics would be helpful before  Ispend time doing it15:12
anteayafair enough15:13
anteayathank you15:13
*** davideagnello has joined #openstack-infra15:13
*** bdemers has quit IRC15:13
anteayawell it sounds like the way br-ex affects the workflow was the kick off to this conversation15:14
anteayaunless it happened earlier and I missed it15:14
anteayaso perhaps more documentation around that?15:14
clarkbsc68cal: maybe what it looks like after devstack runs?15:14
* EmilienM 15:15
*** tonytan4ever has quit IRC15:15
*** shardy has quit IRC15:18
*** changbl has joined #openstack-infra15:18
*** shashank_hegde has joined #openstack-infra15:18
openstackgerritBruno Tavares proposed openstack-infra/puppet-jenkins: Fix race condition in httpd install and config  https://review.openstack.org/22023615:19
*** sidbbt has joined #openstack-infra15:20
fungiclarkb: gah! the new python 3.4 has been removed from trusty-updates because of bug 150076815:21
openstackbug 1500768 in python3.4 (Ubuntu Trusty) "python3.4.3 SRU break requests" [High,Triaged] https://launchpad.net/bugs/150076815:21
clarkbfungi: :/15:21
*** Sree has joined #openstack-infra15:22
fungiapparently this is in part due to ubuntu unbundling urllib3 and shipping a combination of requests and urllib3 which are incompatible with newer python 3.415:22
*** sdake_ is now known as sdake15:23
*** sidbbt has quit IRC15:23
*** hdd has joined #openstack-infra15:23
clarkbnot sure what to think of that15:23
fungiso basically one of the fixes in latest python 3.4 changes behavior in urllib3 which requests relies on (and had patched in their bundled version of it)15:23
fungiwhat a muddy mess15:24
jeblairit's like all of our favorite problems all at once15:24
jeblaira sort of family reunion15:24
openstackgerritDmitriy Ukhlov proposed openstack/requirements: Adds pika RabbitMQ client library  https://review.openstack.org/23101315:24
sc68calclarkb: that would be helpful15:25
*** shardy has joined #openstack-infra15:25
clarkbsc68cal: ok I can work on putting that together. Is the existing doc location good?15:25
*** shardy has quit IRC15:26
sc68calclarkb: yeah existing location is fine15:26
sdaguenotnownikki: does that chase through all the sourcing ?15:26
*** dims_ has quit IRC15:27
sdaguenotnownikki: I guess the question is does that things are late evaluated enough in the bash code we have that I don't know how useful it would be overall, but I'm not opposed15:27
*** rkukura has quit IRC15:27
notnownikkisdague, no, it's just a shallow syntax check on the script you pass to `bash -n`, but it does catch things style checking doesn't like, missing fi on if statements and the like15:28
*** hashar has quit IRC15:29
sdaguenotnownikki: inside functions or just in the main?15:29
*** Ala has joined #openstack-infra15:29
sdagueI thought I tried it and there was goofiness15:29
*** ZZelle has quit IRC15:29
sdagueanyway, if you get something handy that works, I'm all for it15:30
*** Daviey_ has quit IRC15:30
*** Daviey has joined #openstack-infra15:30
*** bpokorny has joined #openstack-infra15:30
notnownikkisdague, it checks everything that is in the file you specify, but not sourced files. So yes, inside functions too15:30
sdaguecool, I say go for it, and ping me when you have patches up15:31
notnownikkisdague, it'll include sample scripts in the tests directory so you can see :)15:31
sdaguecool, great15:32
*** signed8bit has joined #openstack-infra15:32
sdaguelets make sure it works on the current targets as well, devstack, grenade, devstack-gate15:32
*** dims_ has joined #openstack-infra15:32
*** sridhar_ram has joined #openstack-infra15:32
*** geoffarnold has quit IRC15:33
openstackgerritPaul Michali proposed openstack-infra/project-config: Move VPN Functional tests to check queue  https://review.openstack.org/22304915:33
*** geoffarnold has joined #openstack-infra15:34
timrcBeing unemployed is kind of fun.  Maybe I should pick zuul dequeue back up :)15:34
*** scheuran has quit IRC15:34
anteayatimrc: it is15:34
anteayayou get to decide exactly what you want to do with your time15:34
timrcanteaya, It only lasts until Thursday morning :P15:34
anteayaenjoy it while you can15:34
fungidon't forget to change your contributor affiliation to "independent!"15:35
* anteaya had a whole month of on her own time openstack way back when15:35
*** sergmelikyan has quit IRC15:35
timrcfungi, Oh yeah I was dumb and also used my hp.com address for contributions...15:35
*** sergmelikyan has joined #openstack-infra15:35
timrcIIRC this is going to get weird if I switch to my gmail accont, right?15:35
anteayayeah, maybe don't do that starting thursday15:35
timrcSome Ubuntu SSO-ismg.15:35
jeblairno, changing email addresses is fine15:36
jeblairi do it all the time15:36
*** davidlenwell has quit IRC15:36
fungitimrc: just give us a heads up when you first log into gerrit with the new e-mail address and i'll double-check if you ended up with duplicate accounts and fix it right away if so15:36
anteayajeblair: ha ha ha15:36
jeblairyeah, changing the ubuntu sso email is a different beast15:36
fungii think the issue is if you change the e-mail address you're using to authenticate to launchpad.net15:36
fungiright, that15:37
jeblairtimrc: if your launchpad email is a corporate one, you should change that to a personal email15:37
jeblairtimrc: do that once and never change again :)15:37
*** davidlenwell has joined #openstack-infra15:37
jeblairtimrc: but then it's easy to change the address(es) gerrit uses -- ie, the git commit email addresses15:37
*** tjones has quit IRC15:38
fungimainly because, as far as i've seen, launchpad's openid provider doesn't have a way to "change" your e-mail address. it maintains a 1:1 mapping of e-mail address to openid url15:38
*** tjones has joined #openstack-infra15:38
*** krtaylor has quit IRC15:38
fungiwhich is why we've also seen people's duplicate accounts reappear if they alternate netween authenticating with two valid e-mail addresses15:38
timrcjeblair, fungi My login is my gmail account but my "preferred email" is my hp account.15:38
fungitimrc: oh, then you'll be fine15:38
timrcfungi, Awesome.15:39
*** scheuran has joined #openstack-infra15:39
anteayayeah just change your gmail account to preferred, they remove and add others until you are happy, then change your preferred to new job email15:40
anteayaor whatever you like15:40
timrcfungi, Perfect, just logged into review.openstack.org with my gmail.com addr and it did the right thing.  That's a relief.15:40
*** links has quit IRC15:42
*** sshnaidm has quit IRC15:43
*** bdemers has joined #openstack-infra15:43
anteayatimrc: your preferred email is where election ballots and summit passes get sent to15:43
*** Swami has joined #openstack-infra15:44
anteayatimrc: so make sure around ballot and pass distribution time the preferred email is set to something you can access15:44
fungitimrc: you probably also want to make sure your foundation profile is updated to have the same primary e-mail address if you haven't yet15:44
*** geoffarnold is now known as geoffarnoldX15:45
*** sergmelikyan has quit IRC15:46
openstackgerritSean Dague proposed openstack-infra/project-config: remove extra ceilometer devstack plugin declaration  https://review.openstack.org/23155415:46
sdaguefungi / jeblair I think I figured out why ceilometer grenade testing with plugins isn't currently working ^^^^15:47
sdagueit would be nice to get that in to get them unblocked15:47
*** johnsom__ is now known as johnsom15:47
*** bdemers has quit IRC15:48
*** pblaho_ has joined #openstack-infra15:48
*** dtantsur_ has joined #openstack-infra15:48
anteayasdague: +2, can you show me where it is declared in the grenade plugin so I can start checking myself in future?15:48
*** jistr_ has joined #openstack-infra15:48
*** ihrachys_ has joined #openstack-infra15:48
sdagueanteaya: sure thing15:48
openstackgerritMerged openstack/requirements: Require ovs >= 2.4.0.  https://review.openstack.org/22888315:49
*** jtomasek has quit IRC15:49
*** ihrachys has quit IRC15:49
*** naggappan has quit IRC15:49
openstackgerritDmitry Teselkin proposed openstack-infra/project-config: [WIP] rpmlint checks for rpm-packages repo  https://review.openstack.org/22984015:49
*** dtantsur has quit IRC15:49
sdagueanteaya: https://github.com/openstack/ceilometer/blob/71f31b5a14bbc5f74dabce23536a2b27f868735b/devstack/upgrade/settings#L3-L715:49
*** jistr has quit IRC15:50
*** ihrachys has joined #openstack-infra15:50
*** jtomasek has joined #openstack-infra15:50
anteayadoes every project have a devstack/upgrade/settings file?15:50
*** jamesmcarthur has joined #openstack-infra15:50
*** krtaylor has joined #openstack-infra15:51
anteayaor should I just grep for "15:51
anteayadevstack_localrc base enable_plugin"15:51
*** pblaho has quit IRC15:52
*** dtantsur_ has quit IRC15:53
*** ihrachys_ has quit IRC15:53
*** pblaho_ has quit IRC15:53
*** jistr_ has quit IRC15:53
*** pblaho has joined #openstack-infra15:54
*** scheuran has quit IRC15:54
*** geoffarnoldX has quit IRC15:54
anteayasdague: ^15:54
*** dramalho has joined #openstack-infra15:55
*** nadya has quit IRC15:55
*** geoffarnold has joined #openstack-infra15:55
anteayathis is what nova has: http://git.openstack.org/cgit/openstack/nova/tree/devstack/tempest-dsvm-cells-rc15:56
greghaynesclarkb: dsvm nodepool tests were failing oddly for me last night http://logs.openstack.org/14/198114/21/check/gate-dsvm-nodepool/8e291a8/console.html15:57
greghaynesclarkb: I didnt really look into, but you know anything about that?15:57
clarkbgreghaynes: we did hit really slow nodes in hpcloud for a time15:57
clarkbgreghaynes: was it just taking a long time?15:57
anteayasdague: neutron has a devstack/settings file: http://git.openstack.org/cgit/openstack/neutron/tree/devstack/settings15:57
anteayasdague: this is going to be fun15:57
*** dtantsur has joined #openstack-infra15:58
*** unicell has joined #openstack-infra15:58
jeblairclarkb: i have a nodepool-booted ovh node at if you want to take a look15:59
jeblairclarkb: it writes the public ip to all of the files (public and private)15:59
*** mtanino has joined #openstack-infra15:59
*** dramalho has quit IRC15:59
greghaynesclarkb: yea, I think so15:59
*** unicell1 has quit IRC15:59
jeblairclarkb: is that what you were expecting / think that's okay?15:59
clarkbjeblair: ya I think that is expected15:59
*** spzala has quit IRC15:59
clarkbjeblair: basically treat public as private to fallback16:00
*** devvesa has quit IRC16:00
openstackgerritJames E. Blair proposed openstack-infra/nodepool: Handle OVH network configuration  https://review.openstack.org/23156116:00
jeblairclarkb: that 3 line patch is all that's needed to get nodepool (as-is) working with ovh16:00
jeblairfungi: ^16:00
* anteaya has to get a walk in, see y'all in a bit16:01
jeblairgreghaynes: ^16:01
clarkbjeblair: lets go with that then :)16:01
jeblairi've run that locally, and it works with ovh, though i did not do local regression testing against hpcloud/rax16:01
*** dra_ has joined #openstack-infra16:01
*** denisra has quit IRC16:02
jeblairclarkb, fungi, greghaynes: maybe you can take a look at that and let me know if you see any reason you think it might break hpcloud/rax, and if not, i can go ahead and test in production16:02
clarkbjeblair: I would move it up before hpcloud because hpcloud was a fall through before in a case that looks just like ovh and rax16:02
clarkbjeblair: so may be slightly safer with regrad to regressions16:02
jeblairclarkb: hrm, i'm not following -- the way i've written it, neither hpcloud nor rax should make it to the ovh code16:03
jeblairclarkb: oh!16:04
timrcanteaya, All my foundation e-mail was going to my gmail account.16:04
jeblairclarkb: hpcloud will fall through the first time, then it will add a fip16:04
timrcer fungi16:04
jeblairclarkb: that's what you mean, right?16:04
timrcAnd by was I really mean "is"16:04
clarkbjeblair: ya  Ithink it will hit hp as ovh16:06
clarkbjeblair: but if you do ovh and rax first its fine16:06
*** jlanoux has quit IRC16:06
*** shashank_hegde has quit IRC16:06
*** jistr_ has joined #openstack-infra16:06
*** ifarkas has quit IRC16:07
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: fix lp project name to repo name conversion  https://review.openstack.org/23156816:07
*** e0ne has quit IRC16:08
jeblairclarkb: "addresses": {"openstackjenkins2-network5": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:1c:46:ab", "version": 4, "addr": "", "OS-EXT-IPS:type": "fixed"}, {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:1c:46:ab", "version": 4, "addr": "", "OS-EXT-IPS:type": "floating"}]}16:08
jeblairclarkb: i just pulled that from an hpcloud server,16:08
jeblairclarkb: it has no addresses['Ext-Net'], so i think hpcloud will not match the ovh section16:08
clarkbjeblair: and ovh doesn't have private so it won't match hpcloud? if so I think we are fine16:09
*** matrohon has quit IRC16:09
jeblairclarkb: correct16:09
*** AJaeger has joined #openstack-infra16:10
fungiseems like i reviewed a very similar patch to shade16:11
AJaegergood evening16:11
pleia2good morning16:11
AJaegerI have a requirements change for updating constraints for openstackdocstheme. Could I get some reviews, please? https://review.openstack.org/#/c/231325/16:11
greghaynesthat logic LGTM16:11
greghaynesfor nodepool16:12
*** odgrim has quit IRC16:12
*** gyee has joined #openstack-infra16:12
openstackgerritJulia Kreger proposed openstack-infra/shade: WIP: Add logic to support baremetal inspection  https://review.openstack.org/23061516:13
sdagueanteaya: so, only projects with devstack plugins should have devstack/settings files16:13
openstackgerritClint Adams proposed openstack-infra/project-config: Add script to facilitate project renaming in {projects,channels,layout}.yaml  https://review.openstack.org/22800216:13
sdaguedevstack/upgrade/settings is for grenade support16:13
dhellmannjeblair: can you take a look at this zuul-cloner patch? it will make using some of the release tools much nicer. https://review.openstack.org/#/c/220602/16:15
*** geoffarnold has quit IRC16:16
*** geoffarnold has joined #openstack-infra16:16
nibalizergood morning yall16:16
*** Sree has quit IRC16:17
jeblairdhellmann: how will that make things nicer?16:18
openstackgerritClint Adams proposed openstack-infra/project-config: Add script to facilitate project renaming in {projects,channels,layout}.yaml  https://review.openstack.org/22800216:18
*** bdemers has joined #openstack-infra16:19
dhellmannjeblair: because I'll be able to set the environment variable and clone from a local cache instead of over the net. lifeless apparently requires this, and I've found it makes things faster for me, too16:19
*** tmorin has quit IRC16:19
*** mrmartin has joined #openstack-infra16:19
openstackgerritMerged openstack-infra/project-config: remove extra ceilometer devstack plugin declaration  https://review.openstack.org/23155416:20
jeblairdhellmann: you're cloning locally from zuul?  (i'm a little confused)16:20
*** amotoki has quit IRC16:20
dhellmannjeblair: the release tools use zuul-cloner to create a copy of the repo to be modified (tagged, branched, whatever). This option lets us tell zuul-cloner where the local cache is, without requiring release managers to use the same disk layout.16:21
jeblairdhellmann: can you point me at the tools?16:21
openstackgerritMerged openstack-infra/system-config: Update nanliu's git module to puppet-community git  https://review.openstack.org/23139116:21
jeblairdhellmann: (i'm trying to understand the workflow this is designed to support)16:22
dhellmannjeblair: http://git.openstack.org/cgit/openstack-infra/release-tools/tree/make_library_stable_branch.sh16:22
*** austin81 has joined #openstack-infra16:22
dhellmannthat calls clone_repo which is in http://git.openstack.org/cgit/openstack-infra/release-tools/tree/functions16:22
*** links has joined #openstack-infra16:23
*** mrmartin has quit IRC16:23
dhellmannhttp://git.openstack.org/cgit/openstack-infra/release-tools/tree/release_postversion.sh uses the same function16:23
*** jistr_ has quit IRC16:23
timrcnibalizer, Good morning!16:23
dhellmannjeblair: all of this of course presumes someone has checked out a copy of all of our repositories somewhere (or at least the ones interesting for the release tools)16:23
dhellmannjeblair: both lifeless and I do that16:24
timrcOops... my Foundation t-shirt was set to "Small"... that's comical.  Not sure that's statistically the best default :P16:24
fungitimrc: we figured it was a sane default since we're switching the catered lunches to all salad ;)16:26
jeblairdhellmann: in my mind, variables that start with ZUUL_ are ones that are set _by_ zuul (existing ones are: branch, project, url, ref, etc), but we don't anticipated zuul itself setting this one16:26
timrcfungi, lol16:26
jeblairdhellmann: so if we want to do this, we might want to find another name for it?16:26
AJaegerfungi: let's start with the diet early ;)16:27
dhellmannjeblair: that's fine, I was following what I thought was the convention but I see your point. If you want to propose another name, I'll update the patch16:27
timrcfungi, Maybe a number of commits, reviews, LOC per cycle, or a combination of all three can earn you the salad dressing?16:27
openstackgerritMerged openstack-infra/elastic-recheck: Add query for Neutron functional failure bug 1477860  https://review.openstack.org/21847516:27
openstackbug 1477860 in neutron "TestAsyncProcess.test_async_process_respawns fails with TimeoutException" [High,Fix released] https://launchpad.net/bugs/1477860 - Assigned to Jakub Libosvar (libosvar)16:27
*** ildikov has quit IRC16:27
* timrc finds new ways of motivating people to contribute.16:28
jeblairdhellmann: yeah, i'm struggling with this.  give me a bit to process it.  :)16:28
dhellmannjeblair: sure, np, I may be using this tool in a way you hadn't intended :-)16:28
jeblairdhellmann: i think i anticipated z-c being used mostly in a scripted environment -- and your use is, actually...16:28
*** amuller is now known as amuller_afk16:28
*** dramalho has joined #openstack-infra16:29
jeblairdhellmann: i guess you could add an option to release tools, but adding it directly to z-c might solve a more generalized problem.16:29
dhellmannjeblair: yeah, these scripts are eventually going to be invoked in some form from a CI job, so how would that job have normally been configured for this?16:29
*** tsekiyama has joined #openstack-infra16:29
dhellmannjeblair: right, that was fungi's first bit of feedback, too16:29
jeblairdhellmann: most of our invocations of z-c have several command line options, including this one, to tailor it to our ci environment16:30
dhellmannjeblair: maybe ZUUL_CLONER_CACHE_DIR? or ZUULCLONER_CACHE_DIR?16:30
*** mudassirlatif has joined #openstack-infra16:30
dhellmannjeblair: and z-c is being run from the job directly, rather than a script that the job runs?16:30
fungiit varies16:31
*** maishsk has joined #openstack-infra16:31
jeblairdhellmann: both ways, but most of them aren't expected to be run outside of ci16:31
fungithere are some cases where z-c invocation is baked into a job script or possibly a tox env definition16:31
jeblairhaving said that, we are growing some invocations outside of ci, and we've worked on trying to make clean fallbacks there16:32
jeblairso this could be a way of improving that -- updating tooling to set the env variable rather than in command line args16:33
jeblairand in fact, we could actually have zuul send this variable -- that would be an easy way to do that globally :)16:33
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Make oscc_file_contents optional  https://review.openstack.org/23158216:33
jeblairdhellmann: i think i'm really close to talking myself into +2ing the patch as-is.  :)16:34
fungia relevant datapoint is https://git.openstack.org/cgit/openstack/nova/tree/tox.ini#n13 where tox dtrt with a custom envvar but then the jobs use z-c to provide the file in our ci jobs16:34
dhellmannjeblair: should I leave you to talk to yourself, or can I help? :-)16:34
fungier, the job definitions use z-c16:34
*** dimtruck is now known as zz_dimtruck16:34
jeblairfungi: how does z-c fit into that+16:35
fungiso the developer is left to their own devices to put the files into place and tell tox where to find them outside the ci16:35
*** unicell has quit IRC16:35
jeblairfungi: oh i grok16:35
fungijeblair: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n6016:35
*** Daisy has quit IRC16:36
fungibasically an example of "design this so it's flexible for the developer while still allowing us to use z-c in the job"16:36
*** Daisy has joined #openstack-infra16:36
fungias a counterpoint to the local dev tools invoking z-c16:36
*** geoffarnold has quit IRC16:37
jeblairdhellmann: i'm convinced :)16:37
dhellmannI'm also interesting in discussing the idea of making zuul-cloner installable on its own, without the rest of zuul, but I haven't looked at the code to see how much effort that would be16:37
*** nadya has joined #openstack-infra16:37
*** dims_ has quit IRC16:37
dhellmanninstalling zuul isn't that burdensome, but it does bring in some things we don't use at all in the release scripts16:37
*** geoffarnold has joined #openstack-infra16:38
dhellmannjeblair: yay!16:38
*** flashgordon is now known as jogo16:38
jeblairdhellmann: yeah, my thoughts there are that 'the rest of zuul' shouldn't be terribly burdensome... but i guess it brings in gear and statsd (off the top of my head)16:38
openstackgerritMerged openstack-infra/puppet-zuul: Add acceptance tests for puppet-zuul  https://review.openstack.org/21773216:38
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source  https://review.openstack.org/21935116:39
dhellmannjeblair: I count 24 packages installed as dependencies16:39
dhellmannsome may still be needed, of course16:39
dhellmannoops, counted zuul, so make that 2316:39
*** jtomasek has quit IRC16:39
dhellmannanyway, splitting it is much less of a concern, but I've planted the idea16:39
jeblairdhellmann: from the top level: paste, webob, paramiko, python-daemon, statsd, gear, apscheduler are probably zuul-server only16:40
* dhellmann nods16:40
*** maishsk_ has joined #openstack-infra16:40
fungii suppose we could eventually add extras definitions for zuul[server], zuul[merger], zuul[client], zuul[cloner]16:41
*** Daisy has quit IRC16:41
dhellmannoh, I like that16:41
*** Ala has quit IRC16:41
fungis/server/scheduler/ maybe16:41
*** maishsk has quit IRC16:41
*** maishsk_ is now known as maishsk16:41
asselinyolanda, what do you think about https://review.openstack.org/#/c/23158216:41
*** dims_ has joined #openstack-infra16:41
*** dims_ has quit IRC16:41
jeblairoh yeah, i forget about that :)16:42
*** dims_ has joined #openstack-infra16:42
jeblairfungi: 'server' it calls itself16:42
*** claudiub has quit IRC16:42
jeblairmaybe we should rename it16:42
*** jaypipes has joined #openstack-infra16:43
fungiwell, it does still contain some other non-scheduler bits too (api might get spun off?)16:43
yolandaasselin, checking16:43
fungibut yeah "server" is sort of misleading insofar as there are multiple server processes16:43
jeblairhow important are today's image builds? :)16:44
*** dims__ has joined #openstack-infra16:44
*** btully has joined #openstack-infra16:44
*** moravec has left #openstack-infra16:44
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Update pypi metadata to reference -infra  https://review.openstack.org/23158916:44
fungijeblair: i'm not aware of any specific fixes we're waiting on getting baked into images today16:45
yolandaasselin, why not setting that as undef, and add a comparison for undef to create the clouds.yaml file? if not, it will create an empty clouds.yaml right?16:45
fungiso if you accidentally broke/stopped them i wouldn't probably bother trying to restart them16:45
*** dims_ has quit IRC16:45
asselinyolanda, correct.....16:46
jeblairfungi: i haven't done that yet, but i'm thinking of accidentally doing it on purpose16:46
*** rkukura has joined #openstack-infra16:46
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159216:46
jesusaurusyolanda: why undef? can't we just as easily check for ''?16:46
*** moravec has joined #openstack-infra16:47
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159216:48
yolandajesusaurus, nibalizer and crinkle can explain better than me , i've seen latest changes moving from blank to undef values . I need to check if that's for lint, or for tests needs16:49
asselinyolanda, so if it's undef, it's still optional right?16:49
jesusaurusasselin: yeah, it would still be optional if the default is undef16:50
asselinyolanda, jesusaurus I tend to prefer using '' b/c that's a valid user input which can skip creation of the clouds.yaml16:50
crinklejesusaurus: undef is preferred over '' because in puppet 4 the truthiness of '' changes, so instead of going through each module to see if we would be affected we added a linter gem that just prevents it from being used at all16:51
asselinthat said, nodepool works fine with an empty clouds.yaml file16:51
*** ZZelle has joined #openstack-infra16:51
yolandacrinkle, thx, i will remember the reason for that :)16:51
jesusauruscrinkle: ahh16:51
crinkleto be honest i haven't seen a place where it would actually affect us but the gem was added to one module so we kept going with it16:51
yolandai knew there was a reason, but didn't know exactly what, thx16:51
asselinok, so in that case I can check for both undef and ''16:51
asselinor let the empty file get created16:52
*** odgrim has joined #openstack-infra16:52
pabelangerjeblair: in case you missed it: https://review.openstack.org/#/c/231470/ Remove oslo_log dependency for grafyaml16:52
openstackgerrityolanda.robla proposed openstack-infra/puppet-cgit: Add symlink on local git directory  https://review.openstack.org/23045916:53
*** thingee has joined #openstack-infra16:53
yolandaasselin, i'd prefer if the file is not present, than blank16:53
pabelangeranybody know if pbr actually validates the metadata?16:53
*** mwagner_ has quit IRC16:53
jeblairpabelanger: cool16:54
pabelangerunsure if I need to test my metadata changes manually for pypi16:54
*** Sukhdev has joined #openstack-infra16:54
fungipabelanger: it does not, at least not sufficiently. there is a tool... let me see if i can find it16:54
pabelangerfungi: Ya, would be nice to have something in the gate for it16:55
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159216:55
openstackgerritNisha Agarwal proposed openstack/diskimage-builder: Reorder the script number of 'elements/dkms/post-install.d/99-dkms'  https://review.openstack.org/22088516:55
*** odgrim has quit IRC16:55
fungipabelanger: i have it bookmarked somewhere because i thought the same thing but never got around to proposing it for deeper discussion16:56
pabelangerjeblair: oslo_config next16:56
pabelangerfungi: nice! Ya, going to google around and see what is out there16:56
*** armax has quit IRC16:56
*** shashank_hegde has joined #openstack-infra16:57
fungipabelanger: https://warehouse.python.org/project/check-manifest/16:57
fungidstufft suggested it some time ago16:57
*** amuller_afk is now known as amuller16:58
pabelangerfungi: okay, let me see how it works16:58
*** geoffarnold has quit IRC16:58
fungipabelanger: hrm... it doesn't claim to validate metadata16:58
*** geoffarnold has joined #openstack-infra16:59
*** tjones has quit IRC16:59
*** rkukura has quit IRC17:00
*** ybathia has joined #openstack-infra17:00
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Add CI Watch, a third-party CI monitoring dashboard  https://review.openstack.org/23159517:01
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Allow external access by default  https://review.openstack.org/23159617:01
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Allow global ciwatch conf file  https://review.openstack.org/23159717:01
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Fix sqlalchemy models so they work with MySQL backend  https://review.openstack.org/23159817:01
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Remove hardcoded data path  https://review.openstack.org/23159917:01
openstackgerritMikhail S Medvedev proposed openstack-infra/ciwatch: Move configuration step before installation  https://review.openstack.org/23160017:01
fungipabelanger: it's likely this wasn't what i was thinking of after all. i recall discussing at one point a desire to validate setup.cfg for things like invalid trove classifiers (since that will stop a pypi upload cold in its tracks)17:02
*** rkukura has joined #openstack-infra17:02
*** yamamoto has quit IRC17:03
*** cloudtrainme has quit IRC17:03
anteayasdague: okay thanks, is there a list anywhere of projects with devstack plugins?17:03
mmedvedeasselin: I have split out the ciwatch from third-party-ci-tools, rebased on top of new ciwatch repo initial commit, above17:03
* anteaya wonders how to know which projects have them17:03
*** earlephilhower has joined #openstack-infra17:03
anteayatimrc: perfect17:03
*** enikanorov has quit IRC17:03
*** sdake has quit IRC17:03
pabelangerfungi: Ya, that was my concern too. Adding something into the metadata that breaks uploads17:04
asselinmmedvede, yeah just saw that.17:04
sdagueanteaya: we have an unofficial registry that projects can add themselves to17:04
mmedvedeasselin: this way we maintain the history from third-party tools17:04
anteayaand the contractor did a really nice job replacing my stones and promised to _ask_ me if he needed anything from me in future17:04
asselinmmedvede, will review soon17:04
pabelangerfungi: I mean there is testpypi to upload too. But that seems a little overboard17:04
*** cloudtrainme has joined #openstack-infra17:04
sdagueanteaya: http://docs.openstack.org/developer/devstack/plugin-registry.html17:05
anteayasdague: okay great, where might I find this (acknowledging it probablly won't reflect all of them but may be a start)17:05
anteayasdague: thank you17:05
*** sdake has joined #openstack-infra17:05
*** gyee has quit IRC17:05
sdaguethat's generated out of the devstack source tree, so inaccuracies can be patched there17:05
timrcpabelanger, "The main reason17:05
timrcfor this, if OpenStack libraries break, we still want the17:05
timrcinfrastructure to work so we can fix the problem."17:05
timrcShould the infrastructure catch those breakages before they affect the infrastructure :)?17:06
anteayasdague: very good, thank you17:06
*** tonytan4ever has joined #openstack-infra17:06
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Make oscc_file_contents optional  https://review.openstack.org/23158217:06
asselinmmedvede, this is also a good time to update my gerrit queries17:06
*** bharathm has joined #openstack-infra17:06
*** matrohon has joined #openstack-infra17:07
pabelangertimrc: Ya, dogfooding oslo was one of my reasons for using it too.  However, I don't think people are keen on that.17:07
pabelangertimrc: or our responsibility?17:07
timrcI dunno.  I'm just perplexed by the logic.17:07
pabelangertimrc: jeblair and fungi have thoughts on it.  For me, I'm just doing what majority wants :)17:08
pabelangerI liked using it, to learn more about OpenStack17:08
pabelangerbut understand the dependency issues17:08
fungipabelanger: it's mostly that oslo libraries are openstack-specific (that's at least the intent of the "oslo" identifier in them), and infra tools aren't necessarily openstack-specific (they don't benefit from being coinstalled with other openstack services most of the time)17:09
timrcI don't think I'm fit to withstand the combined brain power of jeblair and fungi.  I have no direct experience with oslo* anything myself, but the reasoning seems flawed.  If these Openstack libraries are causing breakages that impact the infrastructure they're getting tested on then what does that say about the infrastructure (and testing)?17:09
*** trown is now known as trown|lunch17:10
jeblairthey add quite a bit of complexity that is needed for openstack and not for our tools17:10
timrcOkay, I think I'm fine with those arguments :)17:10
timrcNot that my feelings matter much here anyway.17:10
jeblair(the patch to remove it is much more red than green)17:10
asselinmmedvede, we need to add some roots to that repo.17:11
pabelangerYa, it does add overhead on the standard logging library, that is for sure17:11
fungiat least the early oslo libraries started life as routines copied from nova into other projects and then later split-out and standardized between services used to provide openstack clouds17:11
*** unicell has joined #openstack-infra17:12
mmedvedeasselin: agreed17:12
asselinfungi, care to add a core to https://review.openstack.org/#/admin/groups/1121,members17:12
fungithey're well adapted to that environment, but often a poor fit for more generalized tools which don't expect the same rarefied environment17:12
pleia2asselin: I can do it17:12
* timrc +1's17:12
asselinpleia2, thanks, we'll start with these folks: https://review.openstack.org/#/admin/groups/688,members17:12
openstackgerritNisha Agarwal proposed openstack/diskimage-builder: Add a new element hpdsa  https://review.openstack.org/22088817:13
pleia2asselin: added you17:13
fungiasselin: pleia2: not an infra project then i guess?17:13
*** mriedem has quit IRC17:13
mmedvedeasselin: also need to add skylerberg17:13
fungiif it is, i need to run the group management script to set up ownership and inclusion correctly, and then formally delegate the members for it17:14
pleia2fungi: oh! right17:14
pleia2fungi: where can I find that script?17:14
fungipleia2: it's in system-config tools17:14
* asselin holds off adding anyone17:14
fungiasselin: yeah, it won't be a self-managed group17:15
jeblairfungi: oops, i accidentally restarted nodepool on purpose.17:15
fungijeblair: thanks for the heads up17:15
pleia2fungi: hm, maintain-infra-groups.py?17:16
fungipleia2: yep17:16
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Expand multinode network documentation.  https://review.openstack.org/23160617:16
*** e0ne has joined #openstack-infra17:16
*** akuznetsova has quit IRC17:16
*** Daisy has joined #openstack-infra17:16
pleia2fungi: is this just run by one of us? anything I should know?17:17
fungiit just runs. i've been running it periodically when i know a new infra project has merged17:17
pleia2ok cool, my turn then17:17
fungidoesn't take any parameters17:17
clarkbsc68cal: 231606 ^ should add some background. I do still need to capture a what the network looks like after devstack as I am not 100% what neutron and nova net do behind the interfaces we give them17:18
clarkbsc68cal: please comment and tell me what is and isn't useful and what needs more documenting17:18
fungipleia2: in summary, it adds infra-core (and potentially infra-puppet-core) as included groups, adds the group to infra-council, and sets infra-ptl as the group owner17:18
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Update pypi metadata to reference -infra  https://review.openstack.org/23158917:19
pleia2fungi: nods, thanks17:19
pleia2it's been run and ciwatch-core was not included, now to figure out why17:19
jeblairpleia2: check governance repo17:20
pleia2jeblair: thanks17:20
jeblairit operates on the list of infra repos there17:20
pleia2yeah, I think that change hasn't merged yet, will confirm17:20
*** geoffarn_ has joined #openstack-infra17:20
fungiyeah, that seems likely. pretty sure i +1'd it but i don't think there's been a tc meeting since17:20
fungiif so it'll likely be merged later today17:21
pleia2yeah https://review.openstack.org/#/c/231181/17:21
*** david-ly_ has joined #openstack-infra17:21
*** david-lyle has quit IRC17:21
openstackgerritJulia Kreger proposed openstack-infra/shade: Update python-troveclient requirement  https://review.openstack.org/23160817:21
pleia2fungi: you still need to +1 :)17:21
*** mriedem has joined #openstack-infra17:21
fungioh, right, it was the corresponding puppet repo i +1'd maybe17:21
openstackgerritMikhail S Medvedev proposed openstack-infra/infra-specs: Correct repository path, spelling  https://review.openstack.org/23160917:22
*** david-ly_ is now known as david-lyle17:22
pleia2asselin: once this lands we'll get the infra users added to this with the script and then we can do the formal delegation17:22
*** erikwilson has joined #openstack-infra17:22
asselinpleia2, ok17:23
*** akuznetsova has joined #openstack-infra17:23
*** sridhar_ram has quit IRC17:23
jeblair| 5263309 | ovh-gra1               | None | ubuntu-trusty-check    | jenkins02 | ubuntu-trusty-check-ovh-gra1-5263309           | ubuntu-trusty-check-ovh-gra1-5263309           | 001c4418-aef2-4576-96e8-58c7dc8bf459 | | ready    | 00:00:03:49 |17:24
*** claudiub has joined #openstack-infra17:24
*** geoffarnold has quit IRC17:24
pabelangerthat looks nice17:24
*** achanda has joined #openstack-infra17:24
jeblairpabelanger: yeah, it takes ovh less than a minute to provision an instance too.  :)17:25
alivigniCan someone look at this patch for me please - https://review.openstack.org/22910017:25
*** harlowja has quit IRC17:25
anteayajeblair: nice17:25
jeblairsomething like 45 seconds17:25
pabelangerjeblair: until -infra starts hammering it :D17:25
jeblairpabelanger: yep, we'll see :)17:25
pabelangervery cool17:25
crinklenibalizer: want to review https://review.openstack.org/#/c/228060/ for me? (it's easy)17:26
*** sabeen1 has quit IRC17:26
*** sabeen1 has joined #openstack-infra17:26
*** sabeen1 has quit IRC17:27
*** moravec has quit IRC17:27
*** sabeen1 has joined #openstack-infra17:27
fungi205097 for gearman-plugin has several +2 votes and no dependencies... is approval being held back for any particular reason? i don't see any review comments to that effect17:27
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use OVH for ubuntu-trusty  https://review.openstack.org/23161217:28
jeblairfungi, clarkb: ^ should we proceed like that?17:28
*** ybathia has quit IRC17:29
clarkbjeblair: ya, you can use the experimental jobs on d-g to test it17:29
*** moravec has joined #openstack-infra17:29
*** dims_ has joined #openstack-infra17:29
*** ybathia has joined #openstack-infra17:29
openstackgerritMonty Taylor proposed openstack-infra/shade: Provide option to delete floating IP with server  https://review.openstack.org/23161417:30
openstackgerritMonty Taylor proposed openstack-infra/shade: Add option to floating ip creation to not reuse  https://review.openstack.org/23161517:30
nibalizercrinkle: in like an hour17:30
openstackgerritMonty Taylor proposed openstack-infra/shade: Add get_server_by_id optmization  https://review.openstack.org/23161617:30
jeblairfungi: i don't think so17:30
clarkbjeblair: oh I would add it to the ubuntu-trusty 2-node label as well17:30
mordredjeblair: morning17:30
clarkboh except only 2 node quota, nodepool should figure that out properly17:30
clarkbup to you17:30
AJaegerdims, dhellmann, sdague: Could you review a constraints update, please? https://review.openstack.org/#/c/231326/17:31
mordredjeblair: looks great17:31
openstackgerritJames E. Blair proposed openstack-infra/system-config: Use OVH for ubuntu-trusty  https://review.openstack.org/23161217:32
dhellmannAJaeger: +2a17:32
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Add get/list/search methods for identity roles  https://review.openstack.org/23161817:32
*** dims__ has quit IRC17:32
jeblairclarkb, mordred: ^ added 2-node17:32
jeblairi think :)17:32
AJaegerdhellmann: thanks!17:32
mordredjeblair: ++17:33
AJaegerdhellmann: could you then +2A also the backport, please? https://review.openstack.org/#/c/231326/17:33
AJaegerdhellmann: and thanks a lot for the release as well!17:33
NobodyCammorning mordred17:33
jeblairclarkb, mordred, fungi: i'm going to speed things up a bit by manually applying that with puppet disabled17:34
mordredjeblair: ++17:34
*** pvaneck has joined #openstack-infra17:34
dhellmannAJaeger: +2a on both (the first link was the backport, but I clicked through the cherry-pick id to the one on master)17:34
*** harlowja has joined #openstack-infra17:35
AJaegerdhellmann: sorry - and thanks17:36
*** cody-somerville has quit IRC17:37
sc68calclarkb: will do17:37
*** dra_ is now known as denisra17:39
*** geoffarn_ has quit IRC17:41
jeblair2015-10-06 17:41:24,618 DEBUG nodepool.NodePool:   ovh-gra1 does not have image ubuntu-trusty for label ubuntu-trusty.17:41
*** geoffarnold has joined #openstack-infra17:41
jeblairgreghaynes, clarkb: that's happening, but it's not mentioned in the missing image check...17:41
jeblairi wonder if we still have a logic problem there17:41
greghaynesjeblair: thats what it looks like17:42
*** armax has joined #openstack-infra17:42
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add controller class and supporting manifests  https://review.openstack.org/23163117:42
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add compute class  https://review.openstack.org/23163217:42
*** fawadkhaliq has quit IRC17:42
greghaynesjeblair: that should be an error not debug, too17:43
*** fawadkhaliq has joined #openstack-infra17:43
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud compute node definition  https://review.openstack.org/21398017:43
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud controller node  https://review.openstack.org/20969817:43
jeblairgreghaynes: well, it's normal during the build/upload process... maybe warning :)17:43
*** doffm has joined #openstack-infra17:43
greghaynesjeblair: oh, I take that back, yea. There are other times we log "could not find image for x" in a similar sitation that are error17:44
jeblairthere are a lot of decision points regarding the missing image check that are not logged17:44
*** abitha has joined #openstack-infra17:45
jeblairmaybe i should write down my thoughts on useful log messages -- one of which is: important decisions should be logged at debug so that program behavior can be reconstructed from logs17:45
doffmI have a gate check failing for a python-novaclient change.17:46
*** maishsk_ has joined #openstack-infra17:46
*** maishsk has quit IRC17:46
*** maishsk_ is now known as maishsk17:46
*** zz_dimtruck is now known as dimtruck17:46
doffmI believe that is because the devstack setup doesn't have those consoles enabled.17:46
*** nadya has quit IRC17:47
openstackgerritMerged openstack-infra/infra-ansible: Switch to Tox for prepping and running tests  https://review.openstack.org/23096617:47
*** abregman has joined #openstack-infra17:47
*** jamesmcarthur has quit IRC17:47
doffmWhere would I find the devstack config that relates to the novaclient functional tests?17:47
*** baoli_ has quit IRC17:47
*** erlon has joined #openstack-infra17:47
*** dtantsur is now known as dtantsur|afk17:48
clarkbdoffm: in openstack-infra/project-config/jenkins/jobs/nova.yaml likely17:48
doffmclarkb: Thanks17:48
clarkbdoffm: that filename might not be correct but the path should be if you grep for functional17:48
mordredjeblair: I agree with you17:48
openstackgerritKaitlin Farr proposed openstack-infra/project-config: Adjust Castellan yaml file  https://review.openstack.org/23163617:49
greghaynesjeblair: that appears to try and grab the snapshot image using the provider image.name, which I think is what we want...17:49
* anteaya looks forward to reading jeblair's thoughts on useful log messages17:49
mordredjeblair: also, because image upload doesn't go through the task interface, we don't log the API calls for it (I noticed that when comaring shade api call counts)17:49
*** jamesmcarthur has joined #openstack-infra17:50
*** glauc____ has joined #openstack-infra17:50
openstackgerritMerged openstack-infra/infra-specs: Correct repository path, spelling  https://review.openstack.org/23160917:50
jeblairmordred: yeah, it's pretty black-box :(17:50
openstackgerritMerged openstack-infra/gearman-plugin: Send node labels back on build completion  https://review.openstack.org/20509717:51
*** dramalho_ has joined #openstack-infra17:51
asselinyolanda, nibalizer this should be what we want now: https://review.openstack.org/#/c/231582/2/manifests/nodepool.pp17:51
*** maishsk has quit IRC17:52
*** dramalho has quit IRC17:52
jeblairgreghaynes, mordred: aha!17:53
*** dims_ has quit IRC17:53
*** glauco_ has quit IRC17:53
jeblairgreghaynes, mordred: if there is a dib image of a given name that is not in the ready state, the missing image check aborts17:53
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Expand multinode network documentation.  https://review.openstack.org/23160617:53
jeblairgreghaynes, mordred: (for that image)17:53
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Expand multinode network documentation.  https://review.openstack.org/23160617:53
jeblairgreghaynes, mordred: basically, it's trying to do "wait until there's a ready one"17:53
clarkbsc68cal: ^ now with diagram of what it looks like after devstack17:54
jeblairgreghaynes, mordred: except the db query is not ordered by time or state, so iff the first row returned is not17:54
jeblairgreghaynes, mordred: except the db query is not ordered by time or state, so iff the first row returned is not-ready, it will abort.  otherwise, it will proceed.17:54
greghaynesjeblair: uhhhh17:54
greghaynesjeblair: how did that ever work17:54
jeblairgreghaynes, mordred: currently, the first row returned is very likely the not-building one17:54
jeblairgreghaynes, mordred: so it's aborting17:55
jeblairgreghaynes: i think it would mostly work most of the time17:55
greghaynesnew project motto17:55
*** regXboi has quit IRC17:55
clarkbgreghaynes: ust like mongodb17:55
jeblairgreghaynes: but yeah, there's some broken logic there, because it's definitely disregarding the fact that there are multiple versions of images17:55
jeblairgreghaynes: it _will_ upload the latest one if it decides to upload one17:56
greghaynes:( two unrelated broken logic finds in this code17:56
jeblairit just won't make that decision in a predictable manner17:56
jeblairgreghaynes: this is in     def checkForMissingDiskImage(self, session, provider, image):17:57
clarkbfungi: yes nested not nesting17:57
jeblairgreghaynes: at             if dib_image.state != nodedb.READY:17:57
anteayaclarkb: should the file be renamed from multinode_diagram to something else?17:57
clarkbfungi: though maybe these are birds we are testing17:57
jeblairgreghaynes, clarkb: i'm going to nodepool dib-image-delete the (not actually) building images17:57
clarkbanteaya: ya should probably be multinode_info or something17:57
clarkbjeblair: ok17:57
fungiclarkb: cool, my comments on ps1 seem to still be relevant to ps317:57
*** jamesmcarthur has quit IRC17:57
clarkbfungi: ya I will collect the comments and push fixes, still need to diagram nova net post devstack17:58
fungioverall made sense17:58
*** moravec has quit IRC18:00
*** cloudtrainme has quit IRC18:00
jeblair2015-10-06 18:00:41,794 INFO nodepool.DiskImageUpdater: Uploading dib image id: 196390 from /opt/nodepool_dib/ubuntu-trusty-1443968159 for template-ubuntu-trusty-1444154441 in ovh-gra118:00
jeblairgreghaynes, clarkb: ^ yup, there we go!18:00
*** dims_ has joined #openstack-infra18:01
greghaynesnice find18:01
*** geoffarnold has quit IRC18:02
*** geoffarnold has joined #openstack-infra18:03
*** yamamoto has joined #openstack-infra18:03
*** spzala has joined #openstack-infra18:03
*** moravec has joined #openstack-infra18:04
*** timcline has joined #openstack-infra18:04
fungiclarkb: not sure if you've seen http://lists.openstack.org/pipermail/openstack-infra/2015-October/003257.html yet. do you have any idea what is actually being requested there? devstack support for vlan trunks on local bridges?18:04
fungithe related neutron/tempest change and lp bug are pretty light on details18:04
*** cloudtrainme has joined #openstack-infra18:05
*** dims_ has quit IRC18:05
*** abregman has quit IRC18:05
clarkbfungi: I have no clue and was thinking about responding "you told us it doesn't work but havne't told us what you need to make it work"18:05
fungiyeah, trying to reverse-engineer the vlan_transparent option in their network tests now18:06
clarkbfungi: It is implied taht ovs does not support what they want to test and ovs is our default neutron setup18:06
*** dims_ has joined #openstack-infra18:06
anteayamind if I ping armax on this?18:06
*** rossella_s has quit IRC18:06
clarkbanteaya: go for it18:06
anteayaas I ping armax on this18:06
*** amuller is now known as amuller_afk18:06
clarkbfungi: its possible we just need to use the linux bridge tests for that particular cse18:06
*** timcline has left #openstack-infra18:06
clarkbfungi: or $other backend18:06
fungiclarkb: that seems likely18:06
fungianteaya: feel free18:07
anteayafungi: thanks :)18:07
*** rossella_s has joined #openstack-infra18:07
*** timcline has joined #openstack-infra18:07
fungiclarkb: right, the e-mail mostly comes across as "i have this change and it's failing tests; can you change the test platform so that my tests pass?"18:07
anteayathat's how I'm reading it18:07
*** timcline has left #openstack-infra18:07
anteayain which case I'd like to punt to armax to follow up18:07
*** yamamoto has quit IRC18:08
anteayadougwig: thank you18:08
fungiindeed, ceasing my thoughts on this may save me time for meeting-day administrivia instead18:08
anteayadougwig: you were my next ping18:08
*** maishsk has joined #openstack-infra18:08
anteayafungi: I support the ceasing18:08
dougwigrandomly noticed this channel.  :)18:08
anteayadougwig: you're awesome18:09
fungi(randomly awesome!)18:09
anteayathe best kind18:10
*** ihrachys has quit IRC18:10
dougwigmy read on that is that the tests they're trying to add need a new job with lb+vlan transparency, all of which can be configured themselves via the standard mechanisms.18:10
anteayadougwig: yay, that is a great read18:11
anteayadougwig: care to do the necessary steps until they hear that message?18:11
*** dramalho_ has quit IRC18:11
dougwigif you mean reply with the path forward, sure.18:12
anteayadougwig: thank you18:12
anteayaand follow up when they can't hear your reply18:13
anteayaat least once18:13
*** claudiub has quit IRC18:13
*** dramalho has joined #openstack-infra18:13
*** links has quit IRC18:13
*** jaypipes has quit IRC18:14
dougwiganteaya: replied.18:15
dougwiganteaya: sledgehammer at the ready.18:15
anteayadougwig: you are my hero18:15
*** dramalho_ has joined #openstack-infra18:15
*** gyee has joined #openstack-infra18:16
*** odgrim has joined #openstack-infra18:16
*** HeOS has quit IRC18:16
openstackgerritMerged openstack-infra/system-config: Use OVH for ubuntu-trusty  https://review.openstack.org/23161218:16
*** peristeri has quit IRC18:16
*** kushal has quit IRC18:17
*** trown|lunch is now known as trown18:17
*** ajmiller has joined #openstack-infra18:17
*** dramalho has quit IRC18:18
bltavareshi there folks, we have this fix waiting for a +2. is someone available to take a look at it? https://review.openstack.org/#/c/225331/18:19
*** BharatK has quit IRC18:19
openstackgerritStefano Maffulli 'reed' proposed openstack-infra/system-config: Removing inactive admins from root access  https://review.openstack.org/15886218:19
*** dramalho_ has quit IRC18:20
*** odgrim has quit IRC18:20
openstackgerritMonty Taylor proposed openstack-infra/shade: Add get_server_by_id optmization  https://review.openstack.org/23161618:20
openstackgerritMonty Taylor proposed openstack-infra/shade: Add option to floating ip creation to not reuse  https://review.openstack.org/23161518:20
*** dramalho has joined #openstack-infra18:23
lifelessdhellmann: I see my name highlighted last night; looks like no action required?18:23
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Expand multinode network documentation.  https://review.openstack.org/23160618:24
clarkbsc68cal: fungi anteaya ^ I think that is ready for merging now18:24
*** cloudtrainme has quit IRC18:24
*** geoffarn_ has joined #openstack-infra18:24
*** ajmiller has quit IRC18:24
*** ayoung_ has joined #openstack-infra18:24
*** odgrim has joined #openstack-infra18:24
*** ajmiller has joined #openstack-infra18:24
*** geoffarnold has quit IRC18:25
anteayaclarkb: fungi's comment in patchset 1 about that and how still stands18:25
clarkbanteaya: I thihought I fixed it18:25
clarkboh different comment18:25
anteayayou did addresses nesting18:26
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Expand multinode network documentation.  https://review.openstack.org/23160618:26
clarkbthere we go18:26
openstackgerritMerged openstack-infra/zuul: Report the per-job build wait time to graphite  https://review.openstack.org/20286718:26
*** sambetts has quit IRC18:26
armaxanteaya: here18:27
* armax scrolls back18:27
*** ivar-lazzaro has joined #openstack-infra18:27
anteayaarmax: dougwig took care of it, thank you though18:28
anteayaarmax: http://lists.openstack.org/pipermail/openstack-infra/2015-October/003257.html18:28
*** abregman has joined #openstack-infra18:28
*** abregman has quit IRC18:28
anteayaarmax: we had a neutron dev confused about how to get tests working18:29
armaxkudos do dougwig18:29
anteayaarmax: thanks :)18:29
funginibalizer: on 158862 it's unclear to me whether that change is wrong or our users_install class is wrong18:29
anteayaclarkb: I like it, thank you18:29
* armax dougwig-beers=+1 armax-beers=018:29
*** olaph has quit IRC18:29
anteayasc68cal: 23160618:30
* anteaya adds +1 to the armax-beers var18:30
anteayaor sake18:30
anteayaarmax: do you like sake?18:30
armaxI do18:30
*** olaph has joined #openstack-infra18:30
anteayawhere is the armax-sake var?18:30
anteayaI'll add to that18:31
sc68calanteaya: +1'd 23160618:31
anteayasc68cal: thank you18:31
anteayathanks clarkb18:31
*** sridhar_ram has joined #openstack-infra18:31
*** mwagner_ has joined #openstack-infra18:31
openstackgerritMonty Taylor proposed openstack-infra/shade: Add get_server_by_id optmization  https://review.openstack.org/23161618:31
openstackgerritMonty Taylor proposed openstack-infra/shade: Add option to floating ip creation to not reuse  https://review.openstack.org/23161518:31
*** sambetts has joined #openstack-infra18:32
*** jamesmcarthur has joined #openstack-infra18:32
*** dramalho has quit IRC18:35
*** cloudtrainme has joined #openstack-infra18:35
reedi was under the impression that one could tell shade an image ID or a name, interchangeably... am I wrong?18:36
*** moravec has quit IRC18:36
mordredreed: yes. you can18:36
reedcode like https://review.openstack.org/#/c/230532/1/firstapp/samples/shade/introduction.py,cm18:36
reedif I put image_id  = 'Ubuntu-14.04' and then pass that to create server, shade barfs18:37
*** geoffarn_ is now known as geoffarnold18:37
reedactually, not shade, it's the cloud that shows middle finger18:37
mordredyah - create_server does not take both - but get_image will18:37
reedOpenStackCloudException: Error in creating instance: Invalid imageRef provided. (HTTP 400) (Request-ID: req-efeca926-e9aa-4de2-8c79-6ac38af99743) (Inner Exception: Invalid imageRef provided.)18:37
mordredreed: add image_id = conn.get_image_id('Ubuntu-14.04')18:38
reedmordred, thanks18:38
mordredreed: sure thing!18:39
reedcalebb, take notice :) ^^18:39
jeblair(uploading to ovh is not the fastest)18:39
clarkbjeblair: its a giant raw image across the atlantic :)18:39
calebbreed: yeah i totally forgot about that18:39
clarkbif we can put a nodepool image builder in ovh that will speed it up18:40
jeblairgreghaynes: !18:40
*** xyang1 has joined #openstack-infra18:40
*** sshnaidm has joined #openstack-infra18:41
jeblairclarkb: i wonder how much slower it really is though18:41
clarkbjeblair: if the window size falls at all the rtt can be deadly18:41
clarkbjeblair: even if we in theory have a giant pipe18:41
*** moravec has joined #openstack-infra18:41
*** hashar has joined #openstack-infra18:42
anteayacan someone check paste.o.o?18:44
anteayaif they have a minute18:44
Shrewsmordred: so, i was holding off on https://review.openstack.org/231029 b/c it failed the nodepool-shade test twice, and I never saw an explanation for that18:44
Shrewsbut, i haven't followed along that closely, either18:44
mordredShrews: me either - but the follow up patch passed the nova test again18:44
anteayain the keystone meetings some folks are noticing it being slow18:44
clarkbanteaya: I think its the same story ...18:44
clarkbwe don't have a robots.txt yet18:44
anteayaclarkb: :(18:45
clarkbgreghaynes: ^ did you see the comments on that? better to just do it in apache18:45
*** geoffarnold has quit IRC18:45
mordredShrews: this one: https://review.openstack.org/#/c/231170/18:45
anteayaanyone know how close we are do getting a robots.txt in place?18:45
greghaynesclarkb: no...18:45
fungianteaya: Clint had a change proposed for it18:45
clarkboh cool18:45
pleia2greghaynes had one too18:45
clarkbgreghaynes: there are problems doin it in the python18:45
*** geoffarnold has joined #openstack-infra18:45
mordredShrews: since _neither_ of those should have had any effects on that test positive or negative, I'm blaming the wind18:45
clarkbpleia2: ya but greghaynes' change is not quite right18:45
Shrewsmordred: weird18:46
clarkbmordred: Shrews that may be the one where hpcloud ws slow18:46
pleia2yeah, I left a long-ish comment ::)18:46
greghaynesyea, if you can not edit the source then thats better18:46
anteayawe have a spike in upload18:46
greghaynesI dont know how we deploy it so not sure how to prevent that18:46
pleia2didn't see Clint's patch, will look18:46
fungiClint's change whitelists the path in the proxy config in apache18:46
fungiif memory serves18:46
greghaynesYea, thats a much better idea18:46
mordredShrews: I made you some new VERY exciting patches though18:46
Shrewsmordred: i'm still working on your ancient ones  :-P18:47
mordredShrews: I hope you like them18:47
pleia2Clint: this is great, thanks18:47
clarkbClint: presumably we need to write the robots.txt somewhere too?18:47
Clintclarkb: pleia2 already did that part18:47
clarkboh thats already there gotcha18:47
Shrewsmordred: https://review.openstack.org/23161818:47
anteayathe patch18:47
greghaynesI also dont think that robots.txt should be the source of the issue here - we should be more than able to handle the load from search engines on paste.o.o18:47
clarkbgreghaynes: its not search engines18:48
greghaynesI think the searching just mighe have exacerbated an existing problem18:48
greghaynesclarkb: what is it from18:48
clarkbgreghaynes: yes its a source of abuse18:48
mordredShrews: https://review.openstack.org/#/c/231614/ and https://review.openstack.org/#/c/231615/ - which I htnk we migh want to surface in ansible when they land18:48
pleia2and spambots like it because it's indexed by search engines, snowball18:48
clarkbwhich may not honor robots.txt but its still step one18:48
mordredShrews: ooh!18:48
greghaynesok, so then two things 1) we should still be able to handle that load I am pretty certain and 2) do we expect spambots to respect robots.txt18:48
*** bswartz has joined #openstack-infra18:49
clarkbgreghaynes: I think we expect some of the bots to respect it and other not to18:49
pleia2greghaynes: if it's no longer indexed by legitimate search engines, it's a much less attractive target18:49
clarkband I have no idea what sort of load lodgeit is expected to handle18:50
anteayaso I'm looking at https://review.openstack.org/#/c/230650/ for the robots.txt patch clint is there another patch?18:50
clarkbits like a single threaded python thing18:50
anteayaClint: or should that fix the issue18:50
pleia2I think our intent here is to try to see if this works :)18:50
Clintanteaya: that's the only extant one18:50
greghaynesYea, thats a fair point. I am still really amazed that we cant keep up with that load (this ran one of the more popular paste sites on the internet a long time ago)18:50
Shrewsmordred: trying to cleanup the reviews needed to get keystone_user module deprecated18:50
anteayapleia2: feel like +A'ing https://review.openstack.org/#/c/230650/18:50
clarkbgreghaynes: its not that surprising, one python process is gonna have a bad time18:50
Shrewsmordred: the last one18:51
pleia2anteaya: still reviewing, never used proxyexclusions before so it's taking me a few minutes18:51
anteayapleia2: ah okay, sorry didn't mean to rush18:51
anteayawasn't sure we were both looking at the same patch18:51
greghaynesclarkb: you can wsgi it if thats the bottleneck18:51
*** wznoinsk has quit IRC18:51
greghaynesclarkb: uwsgi or gunnicorn18:51
clarkbor mod wsgi18:51
greghaynessure, many options ;)18:52
anteayaClint: thank you18:52
* Clint nods.18:52
dhellmannlifeless: I was invoking your name in the discussion of the zuul-cloner change, and jeblair has approved it18:52
*** dramalho has joined #openstack-infra18:54
*** dprince has quit IRC18:55
docaedoI know this is not the devstack channel, but I suspect someone here can answer this question very easily - how do I specify a commit for horizon in devstack?18:55
*** shashank_hegde has quit IRC18:55
*** Sukhdev has quit IRC18:56
clarkbdocaedo: there should be something like HORIZON_REF or HORIZON_BRANCH in stackrc you override that var in localrc to be the ref/commit/tag/branch18:56
jeblairdocaedo: (but also do you know about cross-repo dependencies in zuul (the "Depends-On:" header?))18:56
Shrewsmordred: one of your changes breaks ansible  :(18:56
mordredShrews: ono18:57
Shrewsbreaking APIs is bad, mmmkay18:57
mordredShrews: I agree. let's not break the API then18:57
*** fawadkhaliq has quit IRC18:57
*** fawadkhaliq has joined #openstack-infra18:58
openstackgerritAugustina Ragwitz proposed openstack-infra/project-config: Add script to facilitate project renaming in {projects,channels,layout}.yaml  https://review.openstack.org/22800218:58
docaedoclarkb: thanks, I know HORIZON_BRANCH works for a branch but haven't seen HORIZON_REF18:58
clarkbdocaedo: there is only one, and it should work for all the nameable git things18:58
mordredShrews: UGH. ok18:58
docaedojeblair: I do know about the dependencies more or less :) Specifically want to run devstack with horizon commit before the one that broke a form field for glance18:58
clarkbI just never remember which it is18:58
*** notnownikki has quit IRC18:59
*** jamesmcarthur has quit IRC18:59
mordredShrews: ok. I have an idea of how to do that without breaking API18:59
fungimeeting time!19:00
*** ildikov has joined #openstack-infra19:00
*** bdemers has quit IRC19:00
*** nadya has joined #openstack-infra19:01
*** Daisy has quit IRC19:01
*** Daisy has joined #openstack-infra19:02
*** jamesmcarthur has joined #openstack-infra19:02
openstackgerritMerged openstack-infra/puppet-lodgeit: Serve the local /robots.txt instead of proxying it  https://review.openstack.org/23065019:03
*** bdemers has joined #openstack-infra19:04
*** pelix has quit IRC19:04
*** HeOS has joined #openstack-infra19:05
*** Daisy has quit IRC19:06
*** dizquierdo_ has quit IRC19:09
*** dizquierdo has quit IRC19:09
*** sshnaidm has quit IRC19:10
*** dprince has joined #openstack-infra19:10
*** odgrim has quit IRC19:11
AJaegerdougwig: could you review https://review.openstack.org/#/c/230358/ , please?19:12
dougwigAJaeger: looking19:12
openstackgerritMerged openstack-infra/project-config: Add f22 node matching  https://review.openstack.org/23129819:12
openstackgerritMerged openstack-infra/shade: Update required ironicclient version  https://review.openstack.org/23107019:12
*** flepied has quit IRC19:13
openstackgerritMerged openstack-infra/shade: Update python-troveclient requirement  https://review.openstack.org/23160819:13
AJaegerthanks, dougwig19:13
*** johnsom_ has joined #openstack-infra19:14
dougwigAJaeger: for the stable/liberty override, is there a reason that can't check experimental on a stable patch to do the same thing?19:15
AJaegerdougwig: good question, let me dig a bit...19:15
*** cloudtrainme has quit IRC19:16
openstackgerritMerged openstack-infra/shade: Split get_hostvars_from_server into two  https://review.openstack.org/23102919:17
openstackgerritMerged openstack-infra/shade: Add a private method for nodepool server vars  https://review.openstack.org/23117019:17
openstackgerritMonty Taylor proposed openstack-infra/shade: Add get_server_by_id optmization  https://review.openstack.org/23161619:17
openstackgerritMonty Taylor proposed openstack-infra/shade: Add option to floating ip creation to not reuse  https://review.openstack.org/23161519:17
openstackgerritMonty Taylor proposed openstack-infra/shade: Provide option to delete floating IP with server  https://review.openstack.org/23161419:17
mordredShrews: ^^ fixed the API break. patch is better now19:17
AJaegerdougwig: yeah, that liberty handling is broken, let me -1 - thanks!19:17
*** johnsom has quit IRC19:18
*** ociuhandu has quit IRC19:18
openstackgerritMonty Taylor proposed openstack-infra/shade: Add get_server_by_id optmization  https://review.openstack.org/23161619:18
openstackgerritMonty Taylor proposed openstack-infra/shade: Add option to floating ip creation to not reuse  https://review.openstack.org/23161519:18
openstackgerritMonty Taylor proposed openstack-infra/shade: Provide option to delete floating IP with server  https://review.openstack.org/23161419:18
dougwigAJaeger: ty!19:18
*** cloudtrainme has joined #openstack-infra19:20
pleia2Clint: hey, progress! http://paste.openstack.org/robots.txt19:23
pleia2199.21.84.13 - - [06/Oct/2015:19:19:25 +0000] "GET /robots.txt HTTP/1.1" 404 505 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36"19:23
pleia2-r--r--r-- 1 root root 26 Sep 22 22:05 /srv/lodgeit/openstack/robots.txt19:23
* pleia2 headscratch19:23
*** e0ne has quit IRC19:23
Clintpleia2: hmm, what's DocumentRoot again?19:23
pleia2it's /srv/lodgeit/openstack/, so /srv/lodgeit/${name} should have been right19:24
* Clint frowns.19:24
*** spzala has quit IRC19:26
jeblair2015-10-06 19:25:51,189 INFO nodepool.DiskImageUpdater: Image template-ubuntu-trusty-1444154441 in ovh-gra1 is ready19:27
jeblairclarkb, fungi: ^19:28
pleia2Clint: aha, this shows up when I try to go to it, so maybe it's somewhow looking in the wrong place for robots.txt: [Tue Oct 06 19:28:24 2015] [error] [client] File does not exist: /etc/apache2/htdocs19:29
openstackgerritMerged openstack-infra/project-config: Let magnum jobs to use ceilometer devstack plugin  https://review.openstack.org/23115019:29
pleia2s/shows up/shows up in the apache error.log19:29
*** odgrim has joined #openstack-infra19:29
Clintpleia2: hmm, where's that path coming from?19:30
pleia2I have no idea \o/19:30
*** odgrim has left #openstack-infra19:31
*** e0ne has joined #openstack-infra19:32
anteayaawesome, yay errors19:33
*** rlandy has quit IRC19:33
Clintit's a fun mystery!19:33
bltavarespleia2: what is the name of the file that was created for the proxy configuration?19:35
*** dramalho_ has joined #openstack-infra19:36
pleia2bltavares: the proxy configuration is in the site configuration, /etc/apache2/sites-enabled/10-paste.openstack.org19:36
bltavaresI spent quite a while debugging the proxy configuration once because the file name was not loaded properly19:36
*** ayoung has quit IRC19:36
bltavaresapache was not loading the proxy configuration file because it does not end with a *.conf19:36
*** ayoung_ is now known as ayoung19:37
pleia2the proxy is working, that's why we get a 404 now instead of what we got before19:37
Clintbltavares: this is still apache 2.2, i believe19:37
Clintso the .conf thing doesn't matter yet19:37
bltavaresClint: ah, ok19:37
pleia2indeed it is19:37
lifelessdhellmann: \o/19:37
bltavaresI spent so much time figuring out why my configuration files where not working haha I was expecting that to help19:37
pleia2but yeah, I did the apache2 migration on a bunch of my servers recently, very familiar with the lovely .conf change x_x19:38
*** dramalho has quit IRC19:39
pleia2even knowing about it ahead of time it was a hassle19:39
*** claudiub has joined #openstack-infra19:41
*** cloudtrainme has quit IRC19:41
*** timcline_ has joined #openstack-infra19:42
*** cloudtrainme has joined #openstack-infra19:43
*** bhunter71 has quit IRC19:43
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968519:43
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159219:44
*** wayneeseguin has left #openstack-infra19:45
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968519:46
*** dkranz has quit IRC19:47
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968519:47
*** geoffarnold has quit IRC19:48
Clintpleia2: oh, i don't think there's a DocumentRoot in the vhost config19:48
*** geoffarnold has joined #openstack-infra19:48
*** baoli has joined #openstack-infra19:49
pleia2Clint: aha, you're right19:49
*** dkranz has joined #openstack-infra19:49
*** timcline_ has quit IRC19:49
pleia2lodgeit does magic things, so I don't know that it strictly does the docroot thing?19:49
Clintso we need one19:49
*** timcline has joined #openstack-infra19:49
pleia2for proxypath to work?19:50
Clintjust to serve up that one file19:50
* pleia2 nods19:50
Clintdoesn't have to be in the same dir as the lodgeit stuff19:50
pleia2maybe we create an /srv/www/openstack/ for it19:50
*** nadya has quit IRC19:51
*** tsekiyama has quit IRC19:51
Clintcool, i'll tweak puppet-httpd after the meeting's over19:51
pleia2since I don't want to risk serving up something we should not in /srv/lodgeit/openstack/ anyway19:51
pleia2great, thanks19:51
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968519:51
*** moravec has quit IRC19:52
mordredclarkb: ansible 2.0 beta is out. it's probably worth testing the ansible in d-g with it just to be sure19:52
clarkbmordred: ok, I can whip up a change to install that version from pypi explicitly19:53
*** fawadkhaliq has quit IRC19:53
mordredit should work ... but you know :)19:53
*** fawadkhaliq has joined #openstack-infra19:54
*** shashank_hegde has joined #openstack-infra19:54
clarkbanyone know where the cross project proposals should go? is it too late?19:54
waynranyone aware of issues with JJB that would be problematic on Python 2.7.3?19:54
clarkbI can't find it on the cross project meeting wiki19:54
clarkbwaynr: nope19:54
anteayaclarkb: ttx might know19:55
waynrnevermind, my brain is just desparate for some reason JJB runs are failing in CI but not locally19:55
anteayaclarkb: and the last week's tc logs would have the link19:55
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create bareon project  https://review.openstack.org/23167719:55
*** Sukhdev has joined #openstack-infra19:55
clarkbwaynr: have a link to the failures?19:55
ttxclarkb: odsreg.openstack.org19:56
ttxthere was an email on -dev about it19:56
clarkbmordred: pypi doens't show the beta? is it hiding because its a beta?19:56
anteayattx: thanks19:56
*** spzala has joined #openstack-infra19:56
clarkbmordred: not at http://pypi.dfw.openstack.org/simple/ansible/ eithe19:57
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add non-voting fedora-22 node for puppet-apply tests  https://review.openstack.org/23167919:57
clarkbttx: thanks19:57
*** baoli has quit IRC19:57
*** timcline_ has joined #openstack-infra19:57
*** tsekiyama has joined #openstack-infra19:57
waynrclarkb: https://paste.debian.net/hidden/13f609d8/19:57
jeblairclarkb: you were suggesting an experimental job to test ovh... what was that again?19:58
*** bhunter71 has joined #openstack-infra19:58
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create bareon project  https://review.openstack.org/23167719:58
clarkbjeblair: they are on d-g as experiemntal jobs I forget the exact names but its d-g tmepest full and multinode19:58
openstackgerritPaul Belanger proposed openstack-infra/system-config: Run puppet apply tests with fedora 22  https://review.openstack.org/23168119:58
clarkbjeblair: they run against ubuntu-trusty to test the new image19:58
openstackgerritClint Adams proposed openstack-infra/puppet-httpd: Allow specification of DocumentRoot when reverse-proxying partially  https://review.openstack.org/23168219:58
waynrclarkb: I was thinking it might be a python version thing because operator is a standard library module19:58
Clintor sooner19:59
*** dramalho has joined #openstack-infra19:59
*** kgiusti has left #openstack-infra19:59
pleia2mmmpork: can continue over here :)20:00
anteayaClint: so if you are working on a set of patches related to the same thing, robots.txt in this instance20:00
waynrclarkb: I am using this custom JJB package https://pypi.python.org/pypi/jjb-puppetlabs/1.2.1.dev14520:00
mmmporkpleia2: sure20:00
anteayait helps to select a topic for all related patches20:00
anteayaClint: for instance: https://review.openstack.org/#/q/topic:fix-lodgeit-robots,n,z20:00
pleia2mmmpork: the etherpad we create is essentally our script, let me grab the etherpad from our last rename to help demonstrate...20:00
clarkbwaynr: hrm thats in JJBs override of the yaml parser classes20:00
*** alivigni has quit IRC20:00
anteayaClint: only has one patch, not all of them related to getting robots.txt on paste20:00
*** jaypipes has joined #openstack-infra20:00
Clintanteaya: yeah, i was planning to make them all fix-lodgeit-robots20:01
mmmporki just want to point out, three new folks are assigned to this project and we don't know the things you all know20:01
*** dramalho_ has quit IRC20:01
Clintanteaya: so what are you saying i should do instead?20:01
clarkbwaynr: did you make changes to the yaml parser? there be dragons (I hope we can stp doing that now tha tpyhton2.6 is no longer going to be supported)20:01
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change  https://review.openstack.org/23168320:01
anteayaClint: you can't change the topic of a merged patch20:01
Clintoh, i see20:01
*** timcline has quit IRC20:01
pleia2mmmpork: so back in September we created https://etherpad.openstack.org/p/project-renames-Septemeber-2015 and when it's time to do the renames, root admins hop on the etherpad and chat in IRC to each select tasks (you'll see fungi, pleia2 and clarkb names next to numbered tasks)20:01
anteayaClint: just be mindful that having a topic cross multiple patches if helpful20:01
*** cloudtrainme has quit IRC20:02
pleia2mmmpork: this will be similar, but a much bigger list because we're moving a lot more20:02
anteayaClint: like if there is ahistory to the issue and trying to get a reviewer up to speed on what has already happened20:02
fungimmmpork: basically Clint's been working on a script (you've been reviewing) which creates the change we'll merge to the project-config repo. that will finally be run a day or so before the maintenance to give us all time to review the final output. that takes the place of step #1 at http://docs.openstack.org/infra/system-config/gerrit.html#renaming-a-project20:02
mmmporkpleia2: i went ahead and linked that onto the wiki for the project20:02
*** johnsom_ has quit IRC20:02
waynrclarkb: I am working off a fairly old branch of JJB where I have merged some patches that were still in review at the time20:02
waynrbut I've been using this branch of JJB for many moons20:02
mmmporkfungi: who is going to run the script and when, and then who is going to review? will the change be pushed to gerrit?20:03
*** dramalho has quit IRC20:03
clarkbwaynr: the reason that fails appeas to be you are sorting a nonetype and since None.name does not exist it raises20:03
waynrthe only thing that has really changed is that I am including it using setuptools requirements rather than as a submodule of my config repo20:03
mmmporkfungi: is there a way to run anything else against the patched repo before it's been merged? and what things do we need to check?20:03
clarkbwaynr: has your input changed? eg is the ymal different? maybe you forgot to put a name on a yaml item?20:04
waynrclarkb: that sort is happening on a data structure with 664 elements in it...20:04
pleia2mmmpork: if you look at the etherpad I linked, each repo change had a review.openstack.org change attached to it, historically these were each submitted by a project member in the weeks leading up to it and we review intensely the few days leading up to it20:04
fungimmmpork: i expect Clint will run that script to propose the final change. people who normally review project-config changes would review it just like any project renaming change20:04
pleia2mmmpork: this will only be slightly different because we have Clint's script20:04
mmmporkbut what does "review" mean20:04
waynrand i haven't changed anything about the code that sets the job name20:04
mmmporki guess that's what i'm trying to clarify, are people just looking at the differences manually?20:05
pleia2mmmpork: so take a look at this one: https://review.openstack.org/#/c/201287/20:05
pleia2the one Clint's script will make will be like that, but instead of changing one project, it'll change all of them in one change20:05
mmmporkor is anyone running anything against the name changes to make sure stuff works20:05
fungimmmpork: it means code review. people review changes proposed to the project-config repo just like any of our other git repositories20:05
pleia2so we review it, like we review anything else :)20:05
waynrcertainly not between running it on my jenkins instance and running it locally (the same jenkins-jobs command shown in that paste works when run locally)20:05
mmmporkfungi: yes i understand code review, but does that mean manually look at the code or actually run something against it or...?20:05
openstackgerrityolanda.robla proposed openstack-infra/puppet-cgit: Add symlink on local git directory  https://review.openstack.org/23045920:06
*** ayoung has quit IRC20:06
fungimmmpork: pretty much just look at it. we automate as much cross-checking as we can in our ci already20:06
yolandahi, so i need some more eyes on my changes, tagged with downstream-puppet topic, please20:06
mmmporkok, well if y'all are confident that the cross checking will catch everything, then i'll stop being a pest about it20:07
openstackgerritClint Adams proposed openstack-infra/puppet-lodgeit: Let Apache know where to serve robots.txt from  https://review.openstack.org/23168520:07
nibalizeryolanda: ill try to look today in between conferecning things20:07
yolandaand specially to https://review.openstack.org/#/c/189762/, nodepool secure.conf20:07
clarkbwaynr: right I think theissue is in the input data not the code itself20:07
clarkbwaynr: I think there is a missing name: on a job or template or job group or something20:07
yolandanibalizer, you were the one reviewing those mostly, so i need more eyes :)20:07
Clintanteaya: pleia2: okay, i think that pair might actually fix it20:07
clarkbwaynr: would be easy to make a small test case to see if it reproduces20:07
waynrhmm i'll look into that possibility20:07
*** cloudtrainme has joined #openstack-infra20:07
mmmporki just get nervous when people start talking about broad invasive changes and our verification process is "look at it with human eyes" and "we've done this before so we don't need to worry"20:07
anteayaclarkb: thank you20:07
*** geoffarnold is now known as geoffarnoldX20:07
*** tsekiyama has quit IRC20:08
yolandaleaving for today, good night20:08
*** bhunter71 has quit IRC20:08
pleia2mmmpork: we've been doing renames monthly or bi-monthly for a few years, so we have a pretty solid team of reviewers who know what they're looking for in these changes20:08
pleia2plus, combined with ci that fungi mentioned20:08
nibalizeryolanda: ah20:08
nibalizerhave a good night20:08
waynrclarkb: the input data really hasn't changed between running against this commit of my config repo locally and running against hte same commit in jenkins20:08
*** DericHorn-HP has joined #openstack-infra20:09
fungimmmpork: anything which isn't caught in review gets fixed by the team after the fact. we catch as much as we can through code review and ci automation, but nothing is guaranteed bug-free so we expect to possibly have to fix things afterward20:09
anteayaClint: thank you: https://review.openstack.org/#/q/topic:no-proxy-robots_txt,n,z20:09
*** tsekiyama has joined #openstack-infra20:09
mmmporkpleia2: fungi: so what happens if we break something because we didn't have ci coverage or a review missed something?20:09
fungimmmpork: we fix it20:09
mmmporkother than "someone rapidly patches it"?20:09
Clintanteaya: gotcha20:09
*** geoffarnoldX has quit IRC20:09
mmmporkbut you don't do like a post mortem?20:09
fungimmmpork: why do we need something other than someone rapidly patching it?20:09
anteayaClint: all in one place, friendly for any review who stumbles past20:09
mmmporkif there's a ci gap, it should be fixed20:09
mmmporkif reviewers missed something, maybe think of how to have them not miss that in the future20:10
pleia2our processes do evolve over time as we encounter mistakes or miss things20:10
mmmporkeg, documentation, check list, something?20:10
fungimmmpork: if it's an error which we can find some non-wasteful way to check for, then we add sort of "regression" tests to our ci20:10
*** geoffarnold has joined #openstack-infra20:10
anteayaClint: s/review/reviewer20:10
fungimmmpork: or reminders in the process20:10
mmmporkok, np, i'm just trying to understand what the whole processes are and how we fix mistakes in a bigger sense20:10
mmmporkso if we screw something up in this big change, i just want to make sure we're addressing the larger issue and not just putting in a fast patch each time20:10
fungiwhich as pleia2 points out is how our process has grown to its current state through nearly monthly refining over the past several years20:10
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Do Not Merge Test ansible 2.0 beta  https://review.openstack.org/23168820:11
clarkbmordred: ^ lets see how that does20:11
waynrI wonder if https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning has anything to do with it20:11
fungimmmpork: see for example the change i proposed at https://review.openstack.org/22267020:11
clarkbwaynr: it shouldn't thats just a warning20:12
waynrsince locally I am using Python 2.7.9 but the Jeenkins is using Python 2.7.320:12
clarkbwaynr: we hit it everywhere and it is annoying but shouldn't affect getattr20:12
waynrwould it cause some https requests to fail?20:12
fungimmmpork: that's an "oh and we should remember to check these things next time" addition spotted through trial and error in prior maintenances20:12
clarkbwaynr: nope20:12
waynrthat would lead to a missing 'name'20:12
waynrshoot :(20:12
clarkbwaynr: it proceeds iwth the requiest just fine and yells that it is not secure while doing it20:12
mmmporki'd like us to take some time either in the weekly meeting or seperately to have a brief discussion after we've released this, if there's interest20:13
mmmporkjust to talk about what went well and what we could have done better20:13
mmmporkand if we find holes, make sure they get assigned to someone20:13
mmmporkthat's my biggest concern20:13
anteayawe are doing this so we don't have to do it again20:13
anteayathis is the end20:13
anteayawe hope20:13
mmmporkanteaya: sure, that's fair, but i meant from a bigger picture stance, not just for this issue20:13
fungiwell, except for the occasional projects that want to change their names, which is a lot less common than the namespace moves have been20:14
AJaegeranteaya: there will still be project renames20:14
AJaegerwhat fungi said ;)20:14
*** bswartz has quit IRC20:14
mmmporki mean in general for doing one off things or whatever20:14
anteayaAJaeger: oh sure20:14
mmmporkbut we don't have to if that's not interesting to people20:14
anteayaAJaeger: hopefully once per release20:14
AJaegeranteaya: we have already openstack-infra/puppet-openstack_health scheduled ;(20:14
fungimmmpork: but we also do need to be mindful to keep this fun and not bogged down with more corporate-style change management than absolutely necessary20:14
clarkbnotmyname: b6457e0f95c2563f745bbfb64c739929bc0dc901 is the commit that broke backwrad compat20:14
anteayaactually yeah that was the one I was going to ask about20:15
clarkbnotmyname: it changed the return type of objects20:15
anteayafungi: do you want to wedge teh openstack_health rename in at the same time?20:15
anteayafungi: or wait20:15
fungianteaya: i think we should schedule it for after20:15
clarkbnotmyname: which broke code that iterated over the returned data20:15
anteayafungi: very good20:15
fungianteaya: it's a non-urgent rename20:15
mmmporkfungi: pleia2: thanks for taking the time to address my concerns20:15
clarkbnotmyname: we pinned to 2.3.1 which appeared to be the last release without ObjectBody in it and it worked20:15
fungianteaya: and i'd rather not confuse the stackforge namespace retirement maintenance with unrelated renames20:16
anteayafungi: that was what I was wondering about20:16
anteayafungi: and concur with your choise20:16
*** timcline_ has quit IRC20:16
pleia2mmmpork: I think post-mortems are valuable, but I think as a team we've done a pretty decent job of communicating during these renames, and in the processes around them, which shows in how we make improvements to the process continuously, so we have never felt the need for anything more formal20:16
fungimmmpork: you're welcome! feel free to ask any other questions you have20:16
pleia2mmmpork: but it's certainly something we should be mindful of if this starts to break down20:16
fungialso sorry if i'm not very responsive, i'm trying to follow the tc and cross-project meetings too20:16
clarkbnotmyname: https://review.openstack.org/#/c/229213/ has basic details (its abandoend because we did a larger reqs update at the same time too in another change)20:17
*** timcline has joined #openstack-infra20:17
pleia2mmmpork: and I sympathize with the fact that this makes it hard for newcomers :\20:17
*** aeng has joined #openstack-infra20:17
fungifwiw the rename process used to be _much_ less transparent than it is now20:17
fungi(also much less robust!)20:17
openstackgerritMerged openstack-infra/puppet-infracloud: Initial commit  https://review.openstack.org/22806020:17
pleia2yeah, it's still tedious but openly tedious :)20:18
fungishowing everyone how the sausage is made tends to make it obvious just how tedious ;)20:18
nibalizerthe fith open is open tedium20:18
pleia2Clint: thanks for those changes, I should probably go find some lunch, will have a look in a bit!20:18
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change  https://review.openstack.org/23169220:18
funginibalizer: exactly!20:18
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change  https://review.openstack.org/23169320:18
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change  https://review.openstack.org/23169420:19
Clintpleia2: enjoy!20:19
anteayanibalizer: we all shall have open tedium20:19
anteayafungi: oh and noone talked about the github name changes20:19
anteayafungi: are we all just trying to ignore that part20:19
crinkleSpamapS: greghaynes did you still want to sync up today?20:20
* anteaya pictures hampsters with mice20:20
asselinanteaya, fungi I think github supports a forwarding feature20:20
clarkbmordred: woo looks like ansible2.0 doesn't work20:20
anteayaasselin: do share the docs20:20
mordredclarkb: awesome. I'll dig in and figure out the options20:20
*** ddieterl_ has quit IRC20:20
anteayaasselin: we have never found a way around the gui in the past20:20
* anteaya prays for an update20:20
fungiasselin: yes, we use it (two features actually) but it's not scriptable so we have to click and click until our fingers bleed20:20
clarkbmordred: https://jenkins05.openstack.org/job/gate-tempest-dsvm-f21/169/console its saying command module is not installed20:20
anteayafungi: oh sorry, that is the thing we use20:21
anteayafungi: thanks20:21
clarkbmordred: maybe the install method I used is broken20:21
asselinok, not sure you'll find lots of volunteers for that20:21
*** ayoung has joined #openstack-infra20:21
mordredclarkb: looking at your change20:21
greghaynescrinkle: eh, I dont want to get in your way. I did hear from clarkb that the puppet module versions used in the infra-cloud patchset dont match up with the ones infra uses, so curious what the plan is for that20:21
SpamapScrinkle: I do!20:21
SpamapScrinkle: sitting in on the TC meeting, then after that would probably be a good time20:21
fungianteaya: yeah, you can "rename" a repo in github and you can also "transfer" a repo between orgs in github. they're both process that involves clicking through several web pages and pasting information into form fields and at best it's 20 seconds per rename/transfer20:22
anteayaso if you feel like escalating my permissions for 24 hours so I can participate in the clicking, I'm offering20:22
clarkbI am happy to sit and make my fingers fall off20:22
anteayabut have been turned down before20:22
anteayaso your call20:22
clarkbonce you get in a rythm its not too bad20:23
clarkbjust slow20:23
anteayaclarkb: so cheerful in your tedium20:23
crinklegreghaynes: clarkb the hope is to use something like https://review.openstack.org/#/c/209632/ to install puppet modules locally to each node and run puppet apply so we don't have to worry about conflicts20:23
clarkbassuming I never have to do it again20:23
anteayaclarkb: yay!20:23
clarkbcrinkle: except yo udo have to worry about conflicts because there are shared manifest20:23
clarkbcrinkle: so you have to test that all versions you use work with shared manifest20:23
asselinanteaya, fungi there is a link to 'contact a human' :)20:24
crinkleclarkb: which shared manifest? you mean site.pp?20:24
clarkbcrinkle: the base server stuff20:24
anteayaasselin: thanks, I don't think we want to do that20:24
anteayaasselin: I think we have just accepted that there will be clicking20:24
clarkbcrinkle: which installs users and firewalls and ntp and so on20:24
anteayaasselin: thanks for looking though20:24
clarkbI think ntp was the first one I noticed that may be a problem?20:24
asselinanteaya, why's that?20:24
anteayaasselin: we don't think contacting a human will result in less clicking20:25
crinkleclarkb: I see, I didn't happen to see any conflicts - ntp doesn't change significantly between versions iirc20:25
clarkbcrinkle: well its less the problem today and more the problems tomorrow20:25
nibalizerwe've updated ntp recently, can do so again if we need20:25
clarkbit doubles our test requirements20:25
crinkleclarkb: and i'm pretty sure i just used the same version that's already in modules.env20:25
clarkbfor just about every change20:25
clarkband we are at the mercy of puppet openstack when it ocmes to that20:25
crinkleclarkb: well puppet openstack doesn't really depend on ntp i don't think20:26
clarkbsure so ntp was just an example20:26
clarkbits a generic problem20:26
clarkband unfortunately puppet doesn't make it easy to work around20:26
crinkleclarkb: do you have a suggestion that would make you happier?20:27
crinkleattempting to implement puppet apply was my hope to more or less work around this20:27
clarkbcrinkle: I think we should get a list of deltas to start with and see if it is reasonable to use a common set20:27
*** sdake has quit IRC20:27
nibalizerits not20:27
clarkbnibalizer: what are the deltas?20:28
clarkbapache is no longer a problem20:28
nibalizermysql, end of discussion20:28
clarkband I thought it was the only one20:28
*** sdake has joined #openstack-infra20:28
clarkbnibalizer: eh we don't really use that module anywhere but slave snow20:28
nibalizermaybe i'm being a bit harsh20:28
clarkbso its easy to upgrade20:28
crinkleeverything that is needed for openstack_server is using the same version as defined in modules.env20:28
crinkleyes puppet openstack could update and screw us20:28
clarkbbut my understanding was apache was the problem20:28
clarkband we fixed that20:28
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders no longer use nodedb for diskimages  https://review.openstack.org/22239820:28
clarkbso its weird that we are still working around this20:28
crinkleapache was the most critical problem because it is so old20:29
crinklemysql is also significantly old20:29
nibalizeri figured mysql would be hard enough that we didn't want to mess with it20:29
crinklebut openstack_server doesn't use it20:29
clarkbsure but mysql is easy to update20:29
waynrclarkb: i think the problem is that i messed up creation of my jjb-puppetlabs package20:29
nibalizerwell that changes the game somewhat then20:29
crinklevcsrepo is the other weird one but the infra fork and the puppetlabs one aren't that idfferent20:29
*** fawadkhaliq has quit IRC20:29
waynrpip isn't installing the necessary dependencies for that package when it's installed in Jenkins20:29
clarkblodgeit and test slaves are the only things that use mysql20:29
waynrand the dependencies aren't listed under the project on pypi20:30
openstackgerritMikhail S Medvedev proposed openstack-infra/project-config: Add puppet-ciwatch module to openstack-infra  https://review.openstack.org/23169920:30
clarkbcrinkle: ya and we keep saying we could possibly update vcsrepo anyways. But if that one is out of sync then it is likely the hard one20:30
clarkb(since we use vcsrepo everywhere)20:30
clarkbI bet puppet openstack would just work with the vcsrepo we have20:30
*** pc_m has quit IRC20:31
*** claudiub has quit IRC20:31
openstackgerritDolph Mathews proposed openstack/requirements: Add freezegun to global requirements  https://review.openstack.org/22860320:32
crinkleclarkb: okay, if i understand correctly you would like to correct any version mismatches between what puppet openstack needs and what is defined in modules.env, and then merge https://review.openstack.org/#/c/209617/ into modules.env rather than creating a new list?20:33
clarkbcrinkle: yes20:33
crinkleclarkb: okay20:33
clarkbbecause I think it simplifies testing and I thought that was the entire reason for the apache work we did20:33
crinkleI don't think it was the entire reason but yes apache makes the hurdle smaller20:33
openstackgerritDolph Mathews proposed openstack/requirements: Add freezegun to global requirements  https://review.openstack.org/22860320:34
nibalizerso it  looks like we have mysql::server on paste and wiki20:35
nibalizermaybe a couple other places20:35
nibalizerand the rest are trove?20:35
clarkbnibalizer: yes most things are trove20:35
clarkboh hrm wiki uses it and isn't trove yet? I was not expecting that20:35
nibalizerso my concern is that puts nonimportant work in front of important work20:36
clarkbmordred: looks like the problem is the modules are a git submodule so installing that tar.gz does not work20:37
nibalizerlike I could be wrong, but basically that would mean infracloud progress would have to stop while we fart with mysql20:37
clarkbmordred: so I need to find a proper package for 2.0 beata20:37
clarkbnibalizer: it wouldn't I think you can move forward with the current setup20:37
clarkbnibalizer: but plan to converge them20:37
greghaynesnibalizer: clarkb crinkle The reason I brought this up is I think it is very important we try and make our work on infra-cloud as upstream as possible, and this seemed like a potential hurdle to that20:37
*** baoli has joined #openstack-infra20:37
nibalizerah yes20:37
nibalizerthats my plan20:37
nibalizeror the plan I think we share20:37
nibalizeris to go forward with a separte modules.env file and have 'convrege them' on the todo20:37
greghaynesif we can get to a point where were using more code that is able to land in system-config/etc I think that is a big win20:38
clarkbnibalizer: well doesn't even need to be todo, you can wirte the change and WIP it while you track the necessary steps20:38
*** mtaylor has joined #openstack-infra20:38
nibalizerya so crinkle and I spun up a set of changes to run the apply test on the infracloud manifest with the infracloud modules.env20:38
nibalizerI don't think any of that has landed20:38
nibalizerbut it gives us landable testable code20:39
nibalizerwhich is pretty sweet20:39
asselincrinkle, nibalizer your reviews of these patches would be very much appreciated: https://review.openstack.org/#/q/puppet-openstackci+status:open+owner:%22Ramy+Asselin+%253Cramy.asselin%2540hpe.com%253E%22,n,z20:39
anteayaasselin: thanks for replying to the email, and yes listening to stream events with a personal account is just fine20:39
*** maishsk_ has joined #openstack-infra20:39
*** wolsen has quit IRC20:39
anteayaasselin: and as you point out in the email, posting with a ci account20:39
clarkbnibalizer: it should be self testing?20:39
clarkbor do we need to update some jobs first?20:39
nibalizerclarkb: im not sure what you're asking20:39
*** Nakato has quit IRC20:39
clarkbnibalizer: the change you pushed to test infra cloud manifests against infra cloud module should be self testing20:40
nibalizerclarkb: https://review.openstack.org/#/c/226597/ and it's dependant20:40
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create bareon project  https://review.openstack.org/23167720:40
*** wolsen has joined #openstack-infra20:40
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change.  https://review.openstack.org/23168320:40
clarkbnibalizer: ok so not self testing, I would try and do it in a self testing way20:40
*** Nakato has joined #openstack-infra20:40
clarkbnibalizer: have the apply test do the work20:41
nibalizeris that something the apply test does now?20:41
clarkbnibalizer: no I am saying make the apply test do it without having a separate job20:41
openstackgerritJames E. Blair proposed openstack-infra/devstack-gate: DNM: test change.  https://review.openstack.org/23169220:41
fungiclarkb: wiki and paste/lodgeit may have that class instantiated but are using trove anyway afaik20:41
clarkbnibalizer: so you would update apply test script to just do it20:41
*** maishsk has quit IRC20:41
*** maishsk_ is now known as maishsk20:41
clarkbnibalizer: in the same change you push the module deltas20:42
fungiso we could in theory drop it pretty trivially?20:42
*** mordred has quit IRC20:42
*** mtaylor is now known as mordred20:42
clarkbfungi: if thats the case then ya20:42
*** Sukhdev has quit IRC20:42
fungialso we're close to no longer needing it for slaves too if i find time to finish pushing forward on the node type consolidation to get rid of bare workers20:42
*** dtantsur|afk has quit IRC20:43
*** erlon has quit IRC20:43
*** akuznetsova has quit IRC20:43
*** davidlenwell has quit IRC20:43
*** hdd has quit IRC20:43
*** jhesketh has quit IRC20:43
*** amuller_afk has quit IRC20:43
*** mjb has quit IRC20:43
*** jamielennox has quit IRC20:43
*** zigo has quit IRC20:43
*** mattymo has quit IRC20:43
*** aarefiev has quit IRC20:43
*** amit213 has quit IRC20:43
*** tonyb has quit IRC20:43
*** kuromagi has quit IRC20:43
*** juggler has quit IRC20:43
*** haleyb has quit IRC20:43
*** michchap has quit IRC20:43
*** zz_zz_ja has quit IRC20:43
*** DuncanT has quit IRC20:43
*** krotscheck has quit IRC20:43
*** gus has quit IRC20:43
*** edwarnicke has quit IRC20:43
*** sigmavirus24 has quit IRC20:43
*** eglute has quit IRC20:43
*** d34dh0r53 has quit IRC20:43
*** cloudnull has quit IRC20:43
*** GheRivero has quit IRC20:43
*** j^2 has quit IRC20:43
*** serverascode has quit IRC20:43
*** jraim has quit IRC20:43
*** jroll has quit IRC20:43
*** Trozz has quit IRC20:43
*** dolphm has quit IRC20:43
*** lbragstad has quit IRC20:43
*** comstud has quit IRC20:43
*** jbryce has quit IRC20:43
*** rainya has quit IRC20:43
*** hockeynut has quit IRC20:43
*** zz_gondoi has quit IRC20:43
*** asilenkov has quit IRC20:43
*** tlbr_ has quit IRC20:43
*** mattoliverau has quit IRC20:43
*** vaidy_ril has quit IRC20:43
*** isviridov_away has quit IRC20:43
*** otherwiseguy has quit IRC20:43
*** dtroyer has quit IRC20:43
*** mgagne has quit IRC20:43
*** ruagair has quit IRC20:43
*** TheJulia has quit IRC20:43
nibalizermysql is totally running on paste20:44
clarkbya I expect it there20:44
clarkbwe dropped trove when lodgeit had db connection issues20:44
anteayaha ha ha20:44
anteayapaste and peanut butter20:44
clarkbbut the service may have switched back to trove20:44
anteayaexplains the speed20:44
nibalizerclarkb: so you'd like to see the apply test just become a longer script rather than there be two jobs that give different parameters to the apply test?20:45
*** dtantsur|afk has joined #openstack-infra20:45
openstackgerritPaul Belanger proposed openstack-infra/project-config: Experimental fedora-22 node for puppet-apply tests  https://review.openstack.org/23167920:46
*** kuromagi has joined #openstack-infra20:46
*** fedexo has joined #openstack-infra20:46
*** erlon has joined #openstack-infra20:46
*** akuznetsova has joined #openstack-infra20:46
*** davidlenwell has joined #openstack-infra20:46
*** hdd has joined #openstack-infra20:46
*** jhesketh has joined #openstack-infra20:46
*** amuller has joined #openstack-infra20:46
*** mjb has joined #openstack-infra20:46
*** jamielennox has joined #openstack-infra20:46
*** zigo has joined #openstack-infra20:46
*** mattymo has joined #openstack-infra20:46
*** aarefiev has joined #openstack-infra20:46
*** amit213 has joined #openstack-infra20:46
*** tonyb has joined #openstack-infra20:46
*** juggler has joined #openstack-infra20:46
*** krotscheck has joined #openstack-infra20:46
*** haleyb has joined #openstack-infra20:46
*** michchap has joined #openstack-infra20:46
*** zz_zz_ja has joined #openstack-infra20:46
*** DuncanT has joined #openstack-infra20:46
*** gus has joined #openstack-infra20:46
*** edwarnicke has joined #openstack-infra20:46
*** sigmavirus24 has joined #openstack-infra20:46
*** eglute has joined #openstack-infra20:46
*** d34dh0r53 has joined #openstack-infra20:46
*** cloudnull has joined #openstack-infra20:46
*** GheRivero has joined #openstack-infra20:46
*** j^2 has joined #openstack-infra20:46
*** serverascode has joined #openstack-infra20:46
*** jraim has joined #openstack-infra20:46
*** jroll has joined #openstack-infra20:46
*** Trozz has joined #openstack-infra20:46
*** dolphm has joined #openstack-infra20:46
*** lbragstad has joined #openstack-infra20:46
*** zz_gondoi has joined #openstack-infra20:46
*** comstud has joined #openstack-infra20:46
*** jbryce has joined #openstack-infra20:46
*** rainya has joined #openstack-infra20:46
*** hockeynut has joined #openstack-infra20:46
*** asilenkov has joined #openstack-infra20:46
*** tlbr_ has joined #openstack-infra20:46
*** mattoliverau has joined #openstack-infra20:46
*** vaidy_ril has joined #openstack-infra20:46
*** isviridov_away has joined #openstack-infra20:46
*** otherwiseguy has joined #openstack-infra20:46
*** dtroyer has joined #openstack-infra20:46
*** mgagne has joined #openstack-infra20:46
*** ruagair has joined #openstack-infra20:46
*** TheJulia has joined #openstack-infra20:46
clarkbnibalizer: yes, that has the benefit of being self testing and tests similar things in one test20:46
clarkbnibalizer: agreed paste doesn't have a 3306 connection20:46
clarkbbut does seem to be using a unix socket to local mysqld20:46
*** serverascode has quit IRC20:46
*** fedexo has quit IRC20:46
*** fedexo has joined #openstack-infra20:47
*** pleia2 has quit IRC20:47
*** waynr has quit IRC20:47
*** fawadkhaliq has joined #openstack-infra20:48
*** gildub has joined #openstack-infra20:48
*** Guest70575 has quit IRC20:48
*** trey has quit IRC20:48
fungii guess it never got pointed back to trove after we hunted down the disconnect behavior with review-dev20:49
fungibut should be doable now i think20:49
*** DericHorn-HP has quit IRC20:49
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: Do Not Merge Test ansible 2.0 beta  https://review.openstack.org/23168820:49
clarkbmordred: ^ lets see if that does better20:49
nibalizerif its easy to remove mysql issues I'd love to have us only use one modules.env file20:50
nibalizervcsrepo again should work or be an easy fix20:50
mordredclarkb: OH!20:50
mordredclarkb: you're trying tin stall into a virtualenv20:50
nibalizersome modules are starting to use git_resource for git checkouts now anyways20:50
*** waynr has joined #openstack-infra20:50
clarkbmordred: yes but problem was the tar.gz on github doesn't have the usbmodules I think20:50
fungithere's probably a semi-ancient revert patch to system-config we can rerevert again to move paste.o.o back to trove20:50
jeblairclarkb: it looks like check experimental gets you a lot of jobs that fail :(20:51
clarkbmordred: so cloning recursively and will intsall that20:51
*** trey has joined #openstack-infra20:51
mordredclarkb: k20:51
fungithough we'll need to coordinate another database dump/import when we do20:51
clarkbjeblair: the dibtest ones are the ones you want20:51
*** DericHorn-HP has joined #openstack-infra20:51
clarkbjeblair: I think dvr, enutron sideways, grenade neutron and greande multinode may fail20:51
*** Sukhdev has joined #openstack-infra20:51
*** geoffarnold has quit IRC20:52
*** morgan has quit IRC20:52
jeblairclarkb: oh good, then https://jenkins07.openstack.org/job/experimental-tempest-dsvm-full-dibtest/22/ will do20:52
*** morgan has joined #openstack-infra20:52
*** geoffarnold has joined #openstack-infra20:52
*** sdake has quit IRC20:53
fungialso the experimental nova python27 bindep job uses ubuntu-trusty20:54
armaxhi: I have noticed that the gate-neutronclien-test-dsvm-functional-* have started failing with error20:54
armaxNo hosts matched20:54
armaxan instance here:20:55
clarkbarmax: links?20:55
armaxclarkb: is that something we did20:55
*** Sukhdev has quit IRC20:55
openstackgerritMatthew Treinish proposed openstack-infra/puppet-subunit2sql: Add cron job to run weekly prune the data set of db  https://review.openstack.org/17656420:55
clarkbarmax: no and thats not the problem20:55
armaxclarkb: the patch with the most offending jobs20:55
fungijeblair: feel free to check experimental on https://review.openstack.org/200672 and see what bindep-nova-python27 gives you20:55
armaxclarkb:  https://review.openstack.org/#/c/226424/20:55
*** serverascode has joined #openstack-infra20:55
clarkbarmax: http://logs.openstack.org/24/226424/1/check/gate-neutronclient-test-dsvm-functional-adv-svcs/0836622/console.html#_2015-10-06_19_01_57_887 is where the issue is logged20:55
armaxclarkb: oops you’re right20:56
armaxclarkb: thanks20:56
*** pleia2 has joined #openstack-infra20:56
armaxI didn’t scroll that far back20:56
*** AJaeger has quit IRC20:57
*** Vivek has joined #openstack-infra20:57
jeblairfungi: will do, thanks20:57
*** Rockyg has joined #openstack-infra20:57
*** Vivek is now known as Guest5618820:57
armaxclarkb: ok, we’ll track back to see what’s happening20:57
anteayaclarkb: did you search for exit code in that log?20:57
clarkbanteaya: I just started from top to bottom instead of bottom to top20:58
*** sdake has joined #openstack-infra20:58
*** moravec has joined #openstack-infra20:59
*** jcoufal has joined #openstack-infra20:59
*** dramalho has joined #openstack-infra20:59
jeblairfungi: wow, won the lottery on that one: https://jenkins04.openstack.org/job/bindep-nova-python27/15/20:59
fungiBuilding remotely on ubuntu-trusty-ovh-gra1-526777721:00
notmynameclarkb: just got out of a meeting. thanks for the link21:00
notmynameclarkb: however, there's very little info there (https://review.openstack.org/#/c/229213/)21:00
clarkbnotmyname: yes the problem is the return type chagned21:01
*** DericHorn-HP has quit IRC21:01
*** rfolco has quit IRC21:01
clarkbnotmyname: let me get the code that broke on it21:01
notmynameclarkb: with what call?21:01
*** gyee has quit IRC21:01
*** spzala has quit IRC21:01
anteayais gra1 a region?21:01
notmynameclarkb: also, 2.3.1? we're on 2.6.0. 2.3.1 was release over a year ago21:01
clarkbnotmyname: semver doesn't account got time21:02
fungijeblair: worth noting that failure is expected. today ubuntu yanked teh newer python 2.4 package out of trusty-updates so it's wanting to downgrade21:02
*** waynr has quit IRC21:02
*** waynr has joined #openstack-infra21:02
jeblairfungi: gtk; we can see if it failed correctly :)21:02
anteayabut it will fail successfully21:02
clarkbnotmyname: https://git.openstack.org/cgit/openstack-infra/os-loganalyze/tree/os_loganalyze/generator.py#n158 is where we tripped on it21:02
notmynameclarkb: I'm not arguing that. I'm tryign to figure out what's going on. the fact you were looking at somethign over a year old struck me as odd. I still want to look at it21:02
clarkbnotmyname: AIUI the type changed to a non generator type so we hit the else clause and tried to split a non string type21:03
clarkbnotmyname: self.obj is what swiftcleint returned to us with an _ObjectData21:03
*** dkranz has quit IRC21:04
*** jamesmcarthur has quit IRC21:04
anteayajeblair: is gra1 an ovh region?21:04
clarkbanteaya: yes21:04
anteayaclarkb: thank you21:04
fungier, newer python 3.4 package i meant21:04
anteayait failed21:05
fungiseems like ages since i used python 2.421:05
notmynameclarkb: looking21:05
openstackgerritColleen Murphy proposed openstack-infra/system-config: Update rabbitmq module  https://review.openstack.org/23171321:05
jeblairanteaya: more info here http://docs.openstack.org/developer/os-client-config/vendor-support.html#ovh21:05
*** trown is now known as trown|outttypeww21:05
jeblairfungi: did https://jenkins04.openstack.org/job/bindep-nova-python27/15/console fail in the way you expect?21:06
fungijeblair: basically, yes. it's rare we see a situation where apt-get upgrade breaks because of wanting a package downgrade21:06
fungibut it's a good data point21:06
fungii expect updating the image there will fix it21:07
clarkbfungi: it was just updated today21:07
clarkbhow long ago was new python pulled?21:07
fungiclarkb: yes but the package was also yanked in some recent hours so not sure which happened first21:07
clarkbalso I don't grok how fix for GC would break urllib or requiests21:07
*** fawadkhaliq has quit IRC21:07
anteayajeblair: thank you21:07
fungithough it could also be a transitive resolution problem between main and trusty-updates21:07
*** fawadkhaliq has joined #openstack-infra21:08
jeblairclarkb, fungi: that was probably yesterday's image -- iirc, i killed nodepool before ubuntu-trusty was finished.21:08
fungiclarkb: they "fixed" the gc bug by updating to latest python 3.4.x which included a behavior change (related to fixing previously buggy ssl cert validation)21:08
*** jcoufal has quit IRC21:09
clarkboh right21:09
fungiso when undoing the upload to trusty-updates they basically restored all the bugs which that newer interpreter release was supposed to fix21:09
clarkbthey ended up not going with just the fixes21:09
*** bdemers has quit IRC21:09
anteayajeblair: is the catalyst entry supposed to have that layout? http://docs.openstack.org/developer/os-client-config/vendor-support.html#catalyst21:10
*** burgerk_ has quit IRC21:10
notmynameclarkb: interesting. the response type is still an iterable, but yeah it's no longer a generator21:10
*** aysyd has quit IRC21:10
notmynameclarkb: however, the proposed pin is odd too. the patch that changed it is only in 2.5.0 and 2.6.0. so the pin to <= 2.3.1 (ie skipping 2.4.0) seems odd for this issue21:11
jeblairanteaya: nope, looks like an rst formatting bug21:11
clarkbnotmyname: hrm I checked out the patch and did a git describe on it to find the version21:11
clarkbnotmyname: oh! I bet I didn't do it on the merge commit21:11
clarkbnotmyname: so we can likely update to 2.3.021:12
clarkber 2.4.021:12
anteayajeblair: the source looks right to me: http://git.openstack.org/cgit/openstack/os-client-config/tree/doc/source/vendor-support.rst#n23721:12
*** baoli has quit IRC21:12
anteayait is missing the lines at the bottom of the entry21:12
fungiclarkb: i always use `git tag --contains <whatever>` instead to work around that21:12
jeblairanteaya: yep, on both catalyst and switchengines21:12
anteayajeblair: I'll offer a patch21:12
stevemar_dstufft: poke21:12
jeblairanteaya: thx21:12
*** sharikapongubala has joined #openstack-infra21:13
stevemar_dstufft: not sure if this is possible, but we want to remove https://pypi.python.org/pypi/keystoneauth from pypi, since we are using https://pypi.python.org/pypi/keystoneauth1 instead21:13
*** geoffarnold has quit IRC21:13
stevemar_or is removing from pypi a no-no?21:13
*** baoli has joined #openstack-infra21:14
*** geoffarnold has joined #openstack-infra21:14
*** edmondsw has quit IRC21:14
mordredstevemar_: I mean, that library is there and does exist - I would vote to not delete it21:14
*** changbl has quit IRC21:14
*** anton_khaldin has joined #openstack-infra21:14
stevemar_mordred: but we don't want people using it21:15
mordredstevemar_: we tend to avoid deleting stuff. now - what we might want to do is publish an 0.3.0 of keystoneauth that is empty with a README that says deprecated and a requirement on keystonauth121:15
mordredstevemar_: I can make one of those if you like21:15
stevemar_mordred: how is that any different than deleting, most are unusable :)21:15
mordredstevemar_: 0.2.0 would still be physically there21:15
stevemar_mordred: i never turn down free work :P21:15
*** ldnunes has quit IRC21:16
mordredit would just not be something you'd get accidentally if you weren't trying to get it21:16
stevemar_mordred: yes, that was our next step though, change README to point to keystoneauth1 and cut 0.3.021:16
stevemar_cool beans21:16
stevemar_yeah, if you could do that, i could upvote and remove some of confusion from that page21:17
stevemar_openstacksdk has a similar issue, they were python-openstacksdk at one point21:17
*** fawadk has joined #openstack-infra21:18
*** signed8bit has quit IRC21:20
clarkbarmax: dougwig what is fullstack?21:21
*** dprince has quit IRC21:21
anteayajeblair: https://review.openstack.org/#/c/231718/21:21
armaxclarkb: hang on21:21
armaxclarkb: pulling up references21:21
*** austin81 has quit IRC21:21
armaxclarkb: but in a nutshell21:21
*** fawadkhaliq has quit IRC21:22
armaxclarkb: it’s a set of tests that setup a standalone neutron system, runs a bunch of tests and tear everything down21:22
clarkbarmax: where does it run?21:22
armaxclarkb: http://docs.openstack.org/developer/neutron/devref/development.environment.html#fullstack-tests21:22
armaxclarkb: on neutron changes alone21:22
clarkbarmax: right but part of functional test jobs or unittest jobs or other?21:23
armaxclarkb: for instance: http://logs.openstack.org/85/193485/10/check/gate-neutron-dsvm-fullstack/02bde4e/21:23
clarkbah it is its own job21:24
*** erikmwilson_ has joined #openstack-infra21:24
clarkbis that different than the functional tests?21:24
armaxclarkb: yes, it is21:24
armaxclarkb: its own job21:24
*** bhunter71 has joined #openstack-infra21:24
armaxit’s different from the functional job as it relies on different machinery21:24
*** e0ne has quit IRC21:24
armaxclarkb: I’d defer to amuller if you want to know more21:24
*** wenlock_ has quit IRC21:24
*** baoli has quit IRC21:25
clarkbnah juts making sure I have a basic knoweldge of what it is and how it helps solve the vlan transparency problem21:25
*** matrohon has quit IRC21:25
dougwigclarkb: it's meant to be lighter-weight than dsvm testing, so in theory that job can iterate a bunch of configs instead of just one.21:26
amullerLuckily I've prepared a nice, concise doc (With pictures!) of what fullstack is and why does it exist: http://docs.openstack.org/developer/neutron/devref/fullstack_testing.html21:26
armaxclarkb: yeah, looking into that too but I have the pythonclient fire to put up first21:26
clarkbdougwig: but it runs in devstack looks like?21:26
amullerclarkb: no21:26
*** baoli has joined #openstack-infra21:26
amullerclarkb: I encourage you to read the link I just posted21:26
armaxclarkb: no21:26
clarkbamuller: ok the job says dsvm which usually means yes21:26
clarkbamuller: we should maybe clean that up if not?21:27
amullerThat is a mistake21:27
armaxclarkb: that’s a good point, we should rename that21:27
*** baoli has quit IRC21:27
armaxclarkb: I think that’s vestigial21:27
amullerI'd like to drop the 'dsvm' from the job's name and from ths tox venv yes21:27
amullerfrom the*21:27
clarkbit does run devstack gate21:28
clarkbso we do need it21:28
clarkbthough I wonder if it is running tox if it needs that at all21:28
*** baoli has joined #openstack-infra21:28
*** baoli has quit IRC21:28
* clarkb digs more21:28
*** erikwilson has quit IRC21:28
amullerclarkb: like dsvm-functional it uses devstack to install dependencies21:28
*** erikmwilson_ has quit IRC21:28
amullerclarkb: it has the same dependencies as the neutron functional job if that helps you21:28
*** baoli has joined #openstack-infra21:28
amullerit uses that devstack that installs dependencies but doesn't start the services, I forget the flag's name21:28
openstackgerritFredrik Medley proposed openstack-infra/zuul: Improve job dependencies using graph instead of tree  https://review.openstack.org/23112421:28
clarkbok so it does use devstack21:28
amullerthat devstack flag*21:28
*** hashar has quit IRC21:29
clarkbso the name is correct21:29
*** jcoufal has joined #openstack-infra21:29
*** dizquierdo has joined #openstack-infra21:29
amullerclarkb: yeah I suppose when someone says 'it uses devstack' my brain thinks 'screen', and devstack starting the openstack processes21:29
*** Sukhdev has joined #openstack-infra21:29
*** baoli has quit IRC21:29
clarkbfor us it means "does this thing run stack.sh?"21:29
armaxclarkb: when these jobs were setup, it was faster to rely on devstack for dependency installation21:30
armaxclarkb: but I think the eventual goal was ultimately to move away from that completely21:30
armaxclarkb: I would need to touch base with marun to see how to go about it21:30
clarkbarmax: ya fungi has been working on bindep to make that a possibility21:30
armaxclarkb: yea21:30
clarkbarmax: if al lyou need is things to be installed bindep should handle that well21:30
armaxbindep sounds familiar21:30
armaxand yes, I talked about this with maru in the past21:31
*** baoli has joined #openstack-infra21:31
*** ociuhandu has joined #openstack-infra21:31
fungi"working" is a bit of a poor term there. i've derailed and need to get back to finishing that work21:31
armaxclarkb: to taht effect I don’t think we run stack.sh21:31
* armax checks21:31
fungii have an example job for replacing nova unit tests, but we need a plan for how we can actually transition jobs early in mitaka21:32
fungibecause i'm expecting some probable disruption21:32
clarkbarmax: it does via devstack-gate21:32
clarkbarmax: which is fine and means the name is appropriate21:33
dstufftstevemar_: you can delete it of course, you just need an admin on that package. Though I'd suggest not deleting it and either just directing people to something new, or at a minimum only deleting the releases and not the project itself so that somebody else doesn't come and re-register it21:34
dstufftthough it sounds like you and mordred might have arrived at a different solution, which is fine with me :)21:34
*** baoli has quit IRC21:34
*** geoffarnold has quit IRC21:35
*** geoffarnold has joined #openstack-infra21:35
armaxclarkb: indeed, ok…so we’re good for now21:35
* armax goes to fish through the python-neutronclient failures21:35
armaxclarkb: ping me if you need me21:35
armaxclarkb: unless you want to help me ;)21:35
dstufftGenerally I prefer not deleting, since people may be depending on it and deleting it means they break, but I don't know the specifics of this project if there is anyone likely to be doing that or if it's something we need to care about21:35
clarkbarmax: this is that same fail from before? I can dig in a bit more21:35
anteayajeblair: oh look so much better: http://docs-draft.openstack.org/18/231718/1/check/gate-os-client-config-docs/138039d//doc/build/html/vendor-support.html#catalyst21:36
armaxbasically it complains on the lack of the brctl command21:36
armaxI need to follow the chain of events to see what happened21:36
*** baoli has joined #openstack-infra21:36
armaxlast succseful run was sept 2321:36
clarkbit also trips on remaining_time which is odd21:36
armaxso soemthing must have changed in project-config or in the gate hook that broke that21:37
armaxbut I need to step away for a bit before I can follow the trail properly21:37
jeblairclarkb, fungi: i'm approving the ovh nodepool change since it's been in use most of the day21:38
fungijeblair: cool, go for it. pretty sure i already +2'd it21:38
*** aeng has quit IRC21:39
*** sridhar_ram has quit IRC21:39
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders  https://review.openstack.org/22239821:41
*** baoli has quit IRC21:41
*** ociuhandu has quit IRC21:42
openstackgerritMerged openstack-infra/nodepool: Handle OVH network configuration  https://review.openstack.org/23156121:42
*** baoli has joined #openstack-infra21:42
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968521:42
armaxclarkb: did that ring any bell for you?21:42
*** ayoung has quit IRC21:43
*** annegentle has quit IRC21:43
clarkbarmax: ya I see what is failing but don't understand it yet21:43
*** baoli has quit IRC21:43
clarkbarmax: and I think I know what changed to make it stop working21:43
stevemar_dstufft: cool, i think mordred wants the same deal, so we'll whip-up a new README and point folks to the right spot21:43
clarkbarmax: I1c204d8e91a54df9d39fc6193650cd3e744f1e94 that change21:43
*** baoli has joined #openstack-infra21:44
*** ociuhandu has joined #openstack-infra21:44
* armax looks21:44
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add all production DIB images to OVH  https://review.openstack.org/23172721:44
clarkbarmax: https://git.openstack.org/cgit/openstack/python-neutronclient/tree/neutronclient/tests/functional/hooks/gate_hook.sh#n17 that function apparenyl does not exist despite sourcing the functions file which contains it21:44
*** mriedem has quit IRC21:45
openstackgerritDolph Mathews proposed openstack/requirements: Add freezegun to global requirements  https://review.openstack.org/22860321:45
armaxum, interesting21:45
clarkbarmax: I want to say you can just remove the timeout code because its handled in d-g properly now21:45
*** jamesmcarthur has joined #openstack-infra21:45
clarkbarmax: let me push that patch really quickly21:45
jeblairclarkb, fungi, mordred: i guess this is the next step for ovh? https://review.openstack.org/23172721:45
*** gyee has joined #openstack-infra21:45
armaxclarkb: thanks, I am stepping away for a bit but I’ll be back later21:46
*** harlowja has quit IRC21:46
jeblairclarkb, fungi, mordred: probably we can run that for a bit, check up on jobs that run there, and if we don't see anything amiss, run it up to quota?21:46
*** dizquierdo_ has joined #openstack-infra21:46
*** johnsom has joined #openstack-infra21:46
*** baoli has quit IRC21:47
crinkleclarkb: here are the deltas I can find https://etherpad.openstack.org/p/infra-module-version-deltas21:47
fungijeblair: sounds good21:47
*** baoli has joined #openstack-infra21:47
crinkleclarkb: i'm not worried about concat or vcsrepo, rabbitmq update should be low impact, but mysql is pretty big21:47
pleia2yeesh, mysql will be fun21:48
pleia2I don't know that we're using it in complicated ways though21:48
*** baoli has quit IRC21:48
jeblairi think it may still be used for node setup?21:48
*** johnsom has left #openstack-infra21:48
crinklepleia2: we're not, but the api has changed in the mysql::server class21:48
jeblairthe bindep work should help there21:48
crinklealso we use mysql::python which doesn't exist anymore21:48
pleia2crinkle: ah, good to know21:48
* pleia2 nods21:48
fungiyeah, i need to spend some time thinking about how we can transition all existing jobs that run on bare workers to use the database setup macros (or whether we try to figure out what jobs actually need them?)21:49
*** baoli has joined #openstack-infra21:49
clarkbarmax: https://review.openstack.org/231731 whats odd is that function should be definied so  Istill don't know why it broke but this is at least more correct21:49
* armax saw21:49
*** annegentle has joined #openstack-infra21:50
clarkbcrinkle: we use rabbit somewhere?21:50
anteayawhile I am waiting for paste, Clint how are your patches for robots.txt?21:50
crinkleclarkb: storyboard21:50
clarkboh right21:50
pleia2anteaya: still need to be reviewed21:50
armaxclarkb: thanks, I’ll keep an eye on it21:50
*** wenlock has quit IRC21:52
Clintanteaya: what pleia2 said21:52
anteayapleia2: Clint okay thanks21:52
anteayaI had time to ask, since I was waiting for paste :)21:52
*** julim has quit IRC21:53
*** abitha has quit IRC21:53
clarkbcrinkle: left some notes I think the mysql::python is easy to work around with pymysql21:53
*** abitha has joined #openstack-infra21:53
clarkbcrinkle: we have already largely switched verywhere, checking jeepyb21:54
*** Swami_ has joined #openstack-infra21:54
*** Swami has quit IRC21:55
*** sridhar_ram has joined #openstack-infra21:55
*** aeng has joined #openstack-infra21:55
*** stevemar_ has quit IRC21:55
*** stevemar_ has joined #openstack-infra21:56
clarkbcrinkle: can probably make the mysql-python to pymysql change in jeepyb after the big stackforge move21:56
clarkbcrinkle: I can do that should be straightforward as pymysql is supposed to be compat with mysql-python21:56
*** baoli has quit IRC21:56
crinkleclarkb: cool21:56
*** baoli has joined #openstack-infra21:56
*** harlowja has joined #openstack-infra21:57
*** dramalho has quit IRC21:57
crinkleclarkb: what about nodepool, paste, wiki?21:57
crinklei think those are the ones still using mysql::server not in a separate mysql class21:58
clarkbcrinkle: nodepool already uses pymysql so mysql::python is not necessary, it does use a local db though21:58
clarkbcrinkle: but nodepool should be a non scary one to update21:58
clarkbI think wiki is the only one I am slightly worried about21:58
*** stevemar_ has quit IRC21:58
*** austin81 has joined #openstack-infra21:59
*** gordc has quit IRC22:00
clarkbcrinkle: is it possible to make the necessary mysql changes using the old version first then upgrade? or is there no upgrade path at all?22:01
clarkb(and people think openstack is terrible at upgrading)22:01
*** fawadk has quit IRC22:01
*** glauc____ has quit IRC22:01
crinkleclarkb: I don't think there's an upgrade path, but let me double check versions 1.x and 2.x to make sure22:02
*** armax has quit IRC22:02
crinklei think mysql::server is the only issue, mysql::db and mysql::server::account_security look the same22:02
clarkbcrinkle: if there is a path we can apply it to the slaves first as image builds will fail early and not affect anything if it breaks22:03
clarkbcrinkle: and if that works update everything else properly22:03
*** tonytan4ever has quit IRC22:03
clarkbthen update the version22:03
*** asettle has joined #openstack-infra22:03
openstackgerritDolph Mathews proposed openstack/requirements: Add pep257 as a requirement  https://review.openstack.org/22968522:03
*** weshay has quit IRC22:04
*** Adri2000 has quit IRC22:04
clarkbmordred: looks like ansible 2.0 works if you install it properly22:04
mordredclarkb: awesome22:04
jeblairclarkb: how long should experimental-tempest-dsvm-full-dibtest take?22:06
clarkbjeblair: about an hour22:06
clarkbjeblair: let me pull up a recent run on not gra22:06
*** rossella_s has quit IRC22:06
jeblairclarkb: it's at 1:15 now...22:06
jeblairand going22:06
mordredjeblair: is this our first devstack/tempest on gra?22:06
*** rossella_s has joined #openstack-infra22:06
jeblairmordred: i ran some by hand which were within a few minutes22:07
clarkbjeblair: http://logs.openstack.org/70/229970/3/experimental/experimental-tempest-dsvm-full-dibtest/b482169/console.html thats rax and was 46 minutes22:07
mordredjeblair: cool22:07
clarkbjeblair: I would check if git updates were slow or something22:07
*** stevemar_ has joined #openstack-infra22:07
*** dimtruck is now known as zz_dimtruck22:07
clarkbjeblair: wouldn't be surprised if there is lag in updating things22:07
*** ybathia has quit IRC22:07
*** annegentle has quit IRC22:08
crinkleclarkb: there is not really an upgrade path as far as i can tell, it went through a complete refactor from 1.0.0 to 2.0.0 with no releases in between22:08
*** baoli has quit IRC22:08
*** baoli has joined #openstack-infra22:09
*** edwarnicke has quit IRC22:09
*** mgkwill has quit IRC22:09
*** patrickeast has quit IRC22:09
*** warpc has quit IRC22:09
*** electrical has quit IRC22:09
*** dougwig has quit IRC22:09
*** ams_ has quit IRC22:09
*** sweston has quit IRC22:09
clarkbjeblair: when you did it by hand were you using our image or theirs?22:09
jeblairclarkb: it finished at 1:1922:09
jeblairclarkb: ours22:09
* mordred afks to find the next plane22:09
*** boris-42 has quit IRC22:09
*** davideagnello has quit IRC22:10
anteayamordred: enjoy the next plane22:10
clarkbjeblair: and the run that just happened should use kvm not qemu22:10
jeblairclarkb: setup/devstack/tempest was 15/20/53 minutes22:10
clarkbjeblair: but your manual testing used qemu22:11
clarkbthat would be really weird if it caused the slowness22:11
jeblairclarkb: the rax run was 10/15/35 minutes22:11
clarkbhttp://logs.openstack.org/92/231692/2/experimental/experimental-tempest-dsvm-full-dibtest/3082c43/logs/devstack-gate-setup-workspace-new.txt.gz#_2015-10-06_20_57_46_290 is 4 minutes lost22:12
*** amuller has quit IRC22:12
jeblairclarkb: yeah, though that seems anomalous22:13
*** baoli has quit IRC22:13
waynrclarkb: I can reproduce the issue shown in that paste whenever I use Python 2.7.3, but not on Python 2.7.922:14
*** baoli has joined #openstack-infra22:14
waynrreferring to the JJB problem I was describing earlier22:14
clarkbjeblair: I see 36 minutes22:14
jeblairclarkb: ?22:14
clarkbjeblair: sorry 36 minutes for running the tempest tests22:14
*** mwagner_ has quit IRC22:14
clarkbjeblair: http://logs.openstack.org/92/231692/2/experimental/experimental-tempest-dsvm-full-dibtest/3082c43/console.html#_2015-10-06_22_07_35_04022:14
*** dramalho has joined #openstack-infra22:15
clarkbjeblair: comparing the individual slow test times gra1 is faster per test though but has half the workers22:15
jeblairclarkb: i was counting the tempest phase as the gate hook, that started at 14 past.22:16
clarkbjeblair: rax does tempest in 19 minutes22:16
*** armax has joined #openstack-infra22:16
*** baoli has quit IRC22:16
clarkb53 - 19 is pretty close to 3622:17
jeblairi'm not following your math22:17
clarkbtempest takes 19 minutes on rax http://logs.openstack.org/92/231692/2/experimental/experimental-tempest-dsvm-full-dibtest/3082c43/console.html#_2015-10-06_22_07_35_04022:17
*** mrmartin has joined #openstack-infra22:17
*** geoffarnold has quit IRC22:17
*** geoffarn_ has joined #openstack-infra22:18
jeblairclarkb: what number are you looking at?22:18
clarkband 36 on gra http://logs.openstack.org/92/231692/2/experimental/experimental-tempest-dsvm-full-dibtest/3082c43/console.html#_2015-10-06_22_07_35_04022:18
clarkbjeblair: X tests ran in Y seconds22:18
clarkbsorry used wrong numbers above22:18
clarkbso 36 - 19 = 17 is the different between rax and gra tempest22:18
clarkb53 - 17 = 36 is roughly equal to your numbers that include all the gate hook22:19
clarkbwhich implies the lost time is in tempest22:19
*** baoli has joined #openstack-infra22:19
*** jaypipes has quit IRC22:19
clarkbthe sum of execute time is not wall time its total time and they are roughly the same22:20
clarkbso I think per cpu we get similar performance we just have half th ecpus22:20
jeblairovh: 2015-10-06 22:07:35.040 | Sum of execute time for each test: 2650.3599 sec.22:21
jeblairrax: 2015-10-06 21:23:11.854 | Sum of execute time for each test: 3255.5069 sec.22:21
jeblairthat last was hpcloud22:22
jeblairrax: 2015-10-06 14:59:52.807 | Sum of execute time for each test: 2670.9085 sec.22:22
*** xgerman has quit IRC22:22
*** rackertom has quit IRC22:22
*** Ng has quit IRC22:22
jeblairso yeah, similar wall clock time, for rax/ovh22:22
*** baoli has quit IRC22:22
jeblairer per-test time22:22
*** j^2 has quit IRC22:22
*** baoli has joined #openstack-infra22:23
clarkbthats not wall ist tge sun of the wall times for the workers22:23
jeblairclarkb: so why is the hpcloud sum so much higher while the wall-clock time is about the same? (1283s)22:24
jeblairclarkb: https://jenkins02.openstack.org/job/experimental-tempest-dsvm-full-dibtest/90/consoleFull22:24
jeblairclarkb: there's an hpcloud run i'm looking at22:24
clarkbhpcloud has 8 slower cpus22:24
clarkb4 run the tempest tests so they justvtake more time total22:24
jeblairoh, but we run 4 workers on it22:24
clarkbbut split across twice as many workers wall isnt so bad22:24
jeblairrax also has 4 workers22:25
*** dramalho has quit IRC22:25
jeblairhow many cpus?22:25
clarkbworkerd is cpus/2 min 222:25
jeblairclarkb: i'm still struggling with how rax and hpcloud can have the same runtime but individual tests take longer22:26
jeblair(forget ovh for the moment)22:26
*** mrmartin has quit IRC22:27
*** timcline has quit IRC22:27
*** armax has quit IRC22:28
*** baoli has quit IRC22:28
*** pradk has quit IRC22:28
*** baoli has joined #openstack-infra22:28
*** tlian2 has quit IRC22:31
SpamapSso.. harumph22:31
SpamapSI'm trying to get seutp.cfg to copy some data files from my source tree into the destination system22:31
openstackgerritColleen Murphy proposed openstack-infra/system-config: (WIP) Add Infra Cloud modules to modules.env  https://review.openstack.org/20961722:31
SpamapSbut it just seems to be skipping that part22:32
mtreinishSpamapS: oh, data files, you're in for some fun22:33
jeblairclarkb: well, i still don't understand the math, but do you think we need to ask for 4vcpus?22:33
SpamapSmtreinish: I'm already sort of trying to grok how to find them later with pkg_resources22:34
jeblairclarkb: according to dstat, it looks like the load average spent a while in the 4-6 range in the middle of the tempest run, so seems like it could use it.22:34
SpamapSmtreinish: but at this point, I just want them to be copied in!22:34
mtreinishSpamapS: http://docs.openstack.org/developer/pbr/#files22:34
SpamapSoh thats the missing link right there22:34
mordredSpamapS: you want them in the source tarball or target install filesystem22:35
*** electrical has joined #openstack-infra22:35
jeblairclarkb, fungi, mordred: maybe tomorrow we can find a time to chat about this -- i'm not sure how we want to proceed.22:35
clarkbjeblair those are better questiona for mtreinish22:35
*** annegentle has joined #openstack-infra22:35
clarkbbut my understanding is that tempest doesnt split the tests perfectly22:35
clarkbso total time is 2650 but 2181 of that was in one worker22:36
clarkbthe othrler worker only did 500 seconds of work22:36
mtreinishclarkb: the subunit-trace output shows the worker split pretty well actually22:36
*** Rockyg has quit IRC22:36
mtreinishit's not quite that bad22:36
SpamapSmordred: target filesystem22:36
clarkbso one fix would be to make tempest split that better and do 1325 wall time22:36
mtreinishbut there are some issues22:36
SpamapSpreferrably also available via pkg_resources22:36
jeblairclarkb: i think it's easier if we ignore ovh for now22:36
clarkbmtreinish thats from a run so yes its that bad22:37
mordredSpamapS: data_files is not going to do what you expect22:37
mordredespecially with wheels22:37
SpamapSit looks like I can put things _anywhere_ ?22:37
SpamapSthat doesn't seem right22:37
mtreinishclarkb: http://logs.openstack.org/01/211601/13/gate/gate-tempest-dsvm-full/d079ccd/console.html#_2015-10-06_08_39_27_01522:37
*** jcoufal has quit IRC22:37
*** geoffarn_ is now known as geoffarnoldX22:37
mordredthat is what doesn't work well22:37
mordredSpamapS: put your files in your package dir22:37
mtreinishclarkb: do you have a link?22:38
mordredSpamapS: you got a repo somewhere I can grab?22:38
SpamapSmordred: I'll push22:38
lifelessSpamapS: dstat files :/22:38
*** edwarnicke has joined #openstack-infra22:38
lifelessmordred: ^22:38
*** geoffarnoldX has quit IRC22:38
*** erlon has quit IRC22:38
lifelessI thought we solved this for dib22:38
SpamapSmordred: https://github.com/SpamapS/dstat-plugins22:39
*** geoffarnold has joined #openstack-infra22:39
clarkbmtreinish: http://logs.openstack.org/92/231692/2/experimental/experimental-tempest-dsvm-full-dibtest/3082c43/console.html#_2015-10-06_22_07_35_04022:39
SpamapSI'm entirely o-k with these ending up in /usr/local/share/dstat-plugins/plugins ...22:39
mordredSpamapS: that's hwere they'll wind up22:39
clarkbmtreinish: also those numbers don't add up but its an 11 minute different between workers22:39
SpamapSas long as the 'install-dstat-plugins' script I wrote can find them and install them into wherever the user asks. :)22:39
mordredSpamapS: git mv plugins dstat_plugins ; touch dstat_plugins/plugins/__init__.py ; git add dstat_plugins/plugins/__init__.py22:40
*** ociuhandu has quit IRC22:41
clarkbjeblair: if we ignore ovh what are we trying to solve for?22:41
mordredSpamapS: one sec - pr coming22:41
SpamapSmordred: right now I get them in /plugins .. so.. :-P22:41
SpamapSwell if venv is any indicator22:41
jeblairclarkb: i was just trying to focus on the difference between hp and rax because it eliminates a variable (they have the same number of cpus and workers)22:41
SpamapSwhich it probably isn't22:41
mtreinishclarkb: well everything except for the 2181 sec number is missing setup class and teardown class22:41
clarkbjeblair: yes hpcloud has really slow cpus22:41
*** jamesmcarthur_ has joined #openstack-infra22:41
mtreinishbecause those arent in the subunit22:41
SpamapSmordred: they're .py files.. so putting them under the module isn't a terrible idea. :)22:41
jeblairclarkb: yeah, the numbers still don't add up22:41
clarkbjeblair: so it takes longer22:41
jeblairclarkb: but that's fine22:41
mordredSpamapS: it's a very good idea :)22:42
jeblairclarkb: you're not understanding me22:42
jeblairclarkb: but i don't care22:42
*** jamesmcarthur has quit IRC22:42
*** jamesmcarthur_ is now known as jamesmcarthur22:42
mtreinishthe 2181 sec count is just a time stamp diff to figure out the run time22:42
SpamapSexcept they're weird and won't work without dstat because it imports them oddly22:42
jeblairclarkb: here's what i need to know: do we want to ask ovh for a 4vcpu flavor?22:42
clarkbjeblair: you mean 8vcpu, we have 4 now22:42
clarkbjeblair: do you have a link to the hpcloud numbers?22:43
*** stevemar_ has quit IRC22:43
jeblairclarkb: here is an hpcloud run: http://logs.openstack.org/01/211601/13/gate/gate-tempest-dsvm-full/d079ccd/console.html#_2015-10-06_08_39_27_01522:43
*** warpc has joined #openstack-infra22:43
clarkbso the slowest worker time is 23.5 minutes there and slowest for gra1 is 3622:43
*** stevemar_ has joined #openstack-infra22:43
*** dougwig has joined #openstack-infra22:43
clarkbjeblair: I htink we probably need to22:43
jeblairclarkb: sorry: http://logs.openstack.org/93/231693/1/experimental/experimental-tempest-dsvm-full-dibtest/360f2ec//console.html22:43
mtreinishclarkb: but there are also more workers22:44
clarkbmtreinish: ya so the question is do we want to ask ovh for more cpus22:44
clarkbmtreinish: which would let us run more workers an dI think the answer is yes based on how effective mor ecpus is even when cpu is slower in hpcloud22:44
mordredSpamapS: https://github.com/SpamapS/dstat-plugins/pull/122:44
mtreinishclarkb: yeah, I think that's probably the right way to go22:45
mordredSpamapS: you sure you want Bablel? or is that just from cookiecutter?22:45
mordredSpamapS: foo/lib/python2.7/site-packages/dstat_plugins/plugins/dstat_mysql_io.py22:45
SpamapSmordred: yeah thats just cookiecutter fluff22:45
jeblairclarkb, mtreinish, mordred: okay, so i'll ask for an 8vcpu flavor.  thanks22:46
*** sweston has joined #openstack-infra22:46
*** stevemar_ has quit IRC22:46
SpamapSmordred: ok, and now just import dstat_plugins.plugins and use __file__ to find them.. fun22:47
*** Rockyg has joined #openstack-infra22:47
*** ociuhandu has joined #openstack-infra22:48
jeblairclarkb, mtreinish, mordred: actually, i'd like to be a little more methodical about this, since we've seen both similar and slower run times...22:48
*** ZZelle has quit IRC22:48
jeblairclarkb, mtreinish, mordred: so i think tomorrow i'll try to run more tests and try to get more numbers.22:49
clarkbmordred: ya ansible 2.0 lgtm https://review.openstack.org/#/c/231688/22:49
mordredjeblair: ++22:49
clarkbjeblair: note that the really slow runtimes are for neutron-full in theory becase of the ipv6 tests22:49
clarkbI would need to look at mtreinish's grpahs for nva net22:50
jeblairclarkb: yeah, that's what we benchmarked with and we got similar times with hpcloud22:50
clarkbmtreinish: actually can we get a graph with just tempest-full, multinode full, and neutron-full and neutron multinode?22:50
jeblairclarkb: maybe ovh was actually better at running those and compensated for being slower overall22:50
mtreinishI can generate graphs22:50
mtreinishclarkb: just run time as a line graph? or do you want the box and whiskers?22:51
clarkbsince I think thats the rough set of tests we have current data for with ovh rax and hpcloud22:51
clarkbmtreinish: box and whiskers would be good, do you have by region as well?22:51
mordredSpamapS: pkgutil.get_data('dstat_plugins', 'plugins/dstat_mysql_io.py')22:51
jeblairmtreinish: i would love box and whiskers so i can tell where ovh is landing in the distribution22:51
*** zumbi has joined #openstack-infra22:51
*** ams_ has joined #openstack-infra22:52
mtreinishclarkb: also the numbers in the graphs will correspond to the second run time number in subunit trace (the sum of individual run times)22:52
SpamapSmordred: is there any reason that would be preferred?22:52
mordrednot really22:52
mordredjust letting you know it's a possibility22:52
mtreinishclarkb: I only have it setup to filter on one key in the run_metadata right now22:52
mtreinishso for a fast graph it'll be either build_name or region22:52
mtreinishI can work on doing double grouping but that'll take a bit longer22:52
mordredSpamapS: pkg_resources.resource_filename('dstat_plugins', 'plugins') will get you the path to the dir22:53
*** boris-42 has joined #openstack-infra22:53
*** Rockyg has quit IRC22:53
*** ociuhandu has quit IRC22:53
mordredso you could have your copy script do os.listdir(pkg_resources.resource_filename('dstat_plugins', 'plugins'))22:53
jeblairclarkb: what jobs should we run?  gate-tempest-dsvm-full... maybe gate-nova-python27?22:54
jeblairclarkb: anything else?22:54
openstackgerritlifeless proposed openstack-infra/project-config: Enable constraints proposals for liberty  https://review.openstack.org/23174822:54
lifelessfungi: ^ I have no idea if thats right22:54
jeblairoh, that may not run on a dib node22:54
clarkbjeblair neutron equivalents22:54
clarkbya not dib node but numbers should be roughly comparable22:55
jeblairclarkb: i have to run now, but if you make a list of what's interesting, i'll run them tomorrow22:55
*** annegentle has quit IRC22:55
*** davideagnello has joined #openstack-infra22:56
*** wenlock has joined #openstack-infra22:56
*** jamesmcarthur has quit IRC22:56
SpamapSmordred: Yeah thats what I'll do. It's clearer to humans what the intent is than import + __file__22:56
*** annegentle has joined #openstack-infra22:57
*** dizquierdo_ has quit IRC22:58
*** dizquierdo has quit IRC22:58
*** geoffarnold has quit IRC23:00
*** geoffarnold has joined #openstack-infra23:00
*** cloudtrainme has quit IRC23:01
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud compute node definition  https://review.openstack.org/21398023:01
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add Infra Cloud controller node  https://review.openstack.org/20969823:01
mtreinishjeblair, clarkb: http://i.imgur.com/eOK0g7i.png23:01
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: pyinotify does not work on windows or osx  https://review.openstack.org/23111823:02
mtreinishclarkb: there isn't any data on the multinode jobs in the db because it's nv and on check only23:02
*** mudassirlatif_ has joined #openstack-infra23:02
mtreinishand the db only contains data from runs on the gate pipeline right now23:02
clarkbmtreinish: and whcih number is that again?23:02
anteayamtreinish: the job labels could be larger23:02
clarkbmtreinish: like paste the line otherwise I am sure we will all get confused23:02
*** claudiub has joined #openstack-infra23:03
mtreinishclarkb: "Sum of execute time for each test: 3536.3792 sec."23:03
clarkbmtreinish: ok so its not wall time23:03
clarkbwhich is useful but maybe not super duper useful for what jeblair is trying to do23:03
*** eharney has quit IRC23:03
mtreinishno, the run_time in this case is figured out by just summing the execute time of the tests in the subunit stream23:04
clarkbmtreinish: is it possible to get the longest running subunit stream?23:04
mtreinishI can add support to subunit2sql to give it a run_time value from the cli which we could pass it the wall time23:04
clarkbmtreinish: rather than summing them23:04
clarkbor that23:04
*** mudassirlatif has quit IRC23:05
*** mudassirlatif_ is now known as mudassirlatif23:05
mtreinishclarkb: the issue with passing it the wall time is we need to get that to the gearman worker somehow23:05
*** mgkwill has joined #openstack-infra23:06
mtreinishclarkb: the data for figuring out how long the longest worker took is mostly in the db, it'll take quite a query to figure it out though23:06
*** chlong has quit IRC23:06
mtreinish(the only gaps are around setup and tearDown which aren't in the subunit)23:06
*** baoli has quit IRC23:07
*** wenlock has quit IRC23:07
mtreinishanteaya: and yes the labels are small I added an option to filter the values on output, but the label size isn't adjusted dynamically and the size is set right now assuming it will gonna print like 200 labels23:08
mtreinishit was the guard against the artwork from last week23:08
clarkbjeblair: something like http://paste.openstack.org/show/475531/23:08
SpamapScrinkle: ugh. Running up against the clock here. I guess we'll sync tomorrow.23:09
anteayamtreinish: like we did the other day23:09
anteayamtreinish: ha ha ha23:09
anteayacharlie and the chocolate factory23:09
*** hichihara has joined #openstack-infra23:10
mtreinishanteaya: the nice thing is the resolution is large so you can zoom in pretty well23:10
mtreinishit's also pretty easy to tell which one is neutron23:11
*** ociuhandu has joined #openstack-infra23:11
mtreinishclarkb: wait, what exactly did you mean by the longest running subunit stream? did you mean per worker or just in general (like for a given run: the lastest stop time - the earliest test start time)23:12
clarkbmtreinish: mostly looking for rough wall time23:12
clarkbmtreinish: so if you have 4 workers each with a time the largest time of those 4 is roughly wall time23:13
anteayaha ha ha23:14
anteayayes I did zoom23:14
anteayaneutron stands out in a crowd of test timing results23:14
anteayait is true23:14
crinkleSpamapS: no problem23:14
mtreinishclarkb: well wouldn't even ignoring the workers and just doing max(stop_time) - min(start_time) for all tests in a run give you roughly the same thing23:14
anteayaeven when there is only one other result23:14
mtreinishI think that would be much easier to do23:14
clarkbmtreinish: yes that would work too and you are more familiar with the data so I defer to you23:15
crinkleSpamapS: actually i will be in and out tomorrow for the puppet contributor summit, if i can't make it i can catch up later23:15
*** sigmavirus24 is now known as sigmavirus24_awa23:16
*** ociuhandu has quit IRC23:16
mtreinishclarkb: well the issue with figuring out the workers is that the info is stored in a different table for metadata about a test_run (the execution of a single test during a run)23:16
lifelessclarkb: mtreinish: max(event timestamp) - min(event timestamp) is your overall execution duration23:17
mtreinishit's doable its just getting to be an involved query23:17
lifelesswhat is the specific interest in worker timing ?23:17
*** ociuhandu has joined #openstack-infra23:17
zaroclarkb: ohh man, lots of changes involved in gerrit stream events to always show votes. they all in review now, i'm guessing it  will take a while to propogate thru.23:18
mtreinishclarkb: I can work on adding the max-min query to the db_api and using it for the graph instead that shouldn't be too much work23:18
clarkbzaro: at least the work is up there and upstream seems generally accepting23:18
clarkbmtreinish: neat23:19
mtreinishlifeless: jeblair and clarkb are comparing runs ovh (which has less cpus) to hpcloud23:19
*** shashank_hegde has quit IRC23:19
lifelesszaro: you've fixed the '0 comments don't show up' thing?23:19
*** ociuhandu_ has joined #openstack-infra23:19
mtreinishclarkb: although it'll likely be much slower because it involves the big table23:19
zarolifeless: yes, i believe so23:20
mtreinishbut I guess that doesn't matter to you because you're running via the mtreinish bot :)23:20
mtreinishso it's already high latency23:20
*** Sukhdev has quit IRC23:20
*** rkukura has quit IRC23:21
*** geoffarnold has quit IRC23:21
zarolifeless: https://gerrit-review.googlesource.com/#/c/71051  additional changes in the last comment23:21
*** geoffarnold has joined #openstack-infra23:21
*** hdd has quit IRC23:22
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Add job for the ovs mechanism driver with native mode  https://review.openstack.org/22114323:22
*** ociuhandu has quit IRC23:22
*** ociuhandu_ is now known as ociuhandu23:22
lifelesszaro: \o/\o/\o/23:24
*** fedexo has quit IRC23:24
pleia2it's weird looking at not our gerrit23:24
*** dramalho has joined #openstack-infra23:25
clarkbpleia2: can I get a review on https://review.openstack.org/#/c/231606/ its just a d-g doc change so should be safe23:26
pleia2clarkb: sure23:26
notmynamesdague: I'd like to talk to you about the cross-project service catalog spec23:26
notmynamesdague: do you have some time?23:27
*** j^2 has joined #openstack-infra23:27
clarkbpleia2: it might be easier to read if you pull it locally23:27
pleia2browser is rendering the diagrams like a champ23:29
*** patrickeast has joined #openstack-infra23:29
pleia2this is really good23:29
*** dramalho has quit IRC23:29
pleia2clarkb: nice work, +A23:30
*** mwagner_ has joined #openstack-infra23:33
*** yamahata has joined #openstack-infra23:34
*** hemna is now known as hemnafk23:35
ruagairThat was excellent clarkb.23:35
*** tsekiyama has quit IRC23:35
*** achanda_ has joined #openstack-infra23:36
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest  https://review.openstack.org/23175723:37
*** wenlock has joined #openstack-infra23:37
*** Ng has joined #openstack-infra23:38
*** yamamoto has joined #openstack-infra23:40
*** achanda has quit IRC23:40
anteayanotmyname: morning tends to be sdague's best time23:40
anteayanotmyname: he is usually offline by about 4pm eastern time23:41
*** achanda_ has quit IRC23:41
*** sdake_ has joined #openstack-infra23:41
notmynameanteaya: thanks23:41
anteayanotmyname: welcome23:41
*** sdake has quit IRC23:43
*** geoffarn_ has joined #openstack-infra23:43
*** tsekiyama has joined #openstack-infra23:44
*** xgerman has joined #openstack-infra23:45
*** Daisy has joined #openstack-infra23:46
*** geoffarnold has quit IRC23:46
*** annegentle has quit IRC23:46
*** wenlock has quit IRC23:46
openstackgerritColleen Murphy proposed openstack-infra/system-config: Add baremetal node  https://review.openstack.org/20875123:47
*** yamamoto has quit IRC23:48
*** shashank_hegde has joined #openstack-infra23:49
*** stevemar_ has joined #openstack-infra23:50
openstackgerritColleen Murphy proposed openstack-infra/puppet-infracloud: Add bifrost manifest  https://review.openstack.org/23175723:50
*** rackertom has joined #openstack-infra23:51
*** darrenc is now known as darrenc_afk23:51
*** sdake_ has quit IRC23:52
*** xyang1 has quit IRC23:52
*** Swami_ has quit IRC23:53
*** Daisy has quit IRC23:53
greghaynescrinkle: puppet-infraclout?23:54
*** Daisy has joined #openstack-infra23:54
greghayneser, infracloud23:54
*** tsekiyama has quit IRC23:54
openstackgerritgreghaynes proposed openstack-infra/nodepool: Fix check for missing disk image  https://review.openstack.org/23176023:55
*** mestery has quit IRC23:56
greghaynesjeblair: ^ I think that fixes the bug from earlier23:56
crinklegreghaynes: i think it makes sense to have a module for this, do you disagree?23:56
*** tsekiyama has joined #openstack-infra23:56
greghaynescrinkle: I have no idea23:56
greghaynescrinkle: I had no idea it existed23:56
*** mestery has joined #openstack-infra23:56
crinklegreghaynes: it is new23:56
openstackgerritMerged openstack/requirements: Update constraints for openstackdocstheme 1.2.3  https://review.openstack.org/23132523:57
*** yamahata has quit IRC23:57
*** ajmiller has quit IRC23:57
*** Daisy has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!