| lifeless | is that a morange? | 00:02 |
|---|---|---|
| nibalizer | maybe meringue | 00:02 |
| *** devkulkarni has quit IRC | 00:04 | |
| *** abregman|ZzZz has quit IRC | 00:06 | |
| *** hemna is now known as hemnafk | 00:07 | |
| *** tlian has quit IRC | 00:07 | |
| *** tiswanso has joined #openstack-infra | 00:09 | |
| *** madong_ has joined #openstack-infra | 00:10 | |
| *** larainema has quit IRC | 00:10 | |
| *** rlandy has quit IRC | 00:12 | |
| *** darrenc_afk is now known as darrenc | 00:14 | |
| *** mtanino has quit IRC | 00:15 | |
| *** tsekiyama has quit IRC | 00:16 | |
| *** glauco has joined #openstack-infra | 00:16 | |
| *** pvaneck has quit IRC | 00:17 | |
| *** achanda has quit IRC | 00:18 | |
| *** patrickeast has quit IRC | 00:18 | |
| *** tjones has quit IRC | 00:18 | |
| *** Sukhdev_ has quit IRC | 00:19 | |
| harlowja | howdy folks, i have yet another request to replace the 0.7.x branch for stackforge cloud-init with the newest conversion from the old launchpad repo (that is still getting some changes) | 00:21 |
| *** tjones has joined #openstack-infra | 00:21 | |
| *** abdelwas has quit IRC | 00:21 | |
| harlowja | i know its a pita to do this, but it would be very much appreciated to do it (again) | 00:22 |
| harlowja | https://github.com/harlowja/cloud-init/tree/0.7.x (is the resync) | 00:22 |
| mordred | harlowja: is the gerrit repo going to be the place where things happen after this? | 00:22 |
| mordred | harlowja: like, as soon as we do a resync, the old repo needs to diaf | 00:22 |
| *** thorst1 has joined #openstack-infra | 00:22 | |
| *** achanda has joined #openstack-infra | 00:22 | |
| harlowja | https://github.com/stackforge/cloud-init/ | 00:22 |
| *** jasondotstar has quit IRC | 00:23 | |
| *** AzherKhan has joined #openstack-infra | 00:23 | |
| harlowja | yes, but some development still keeps on getting merged into the bzr stuff :-/ | 00:23 |
| harlowja | (not due to me afaik) | 00:23 |
| mordred | right. I think what i'm saying is that the bzr repo needs to be deleted | 00:23 |
| mordred | I'm happy to resync this for you one more time - but I'm not happy to resync it more than that | 00:23 |
| *** thorst1 has quit IRC | 00:24 | |
| openstackgerrit | Merged openstack-infra/puppet-cgit: Boilerplate beaker-rspec files https://review.openstack.org/206372 | 00:24 |
| openstackgerrit | Merged openstack-infra/puppet-diskimage_builder: Boilerplate beaker-rspec files https://review.openstack.org/206373 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-drupal: Boilerplate beaker-rspec files https://review.openstack.org/206374 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-elastic_recheck: Boilerplate beaker-rspec files https://review.openstack.org/206375 | 00:25 |
| *** sputnik13 has quit IRC | 00:25 | |
| openstackgerrit | Merged openstack-infra/puppet-elasticsearch: Boilerplate beaker-rspec files https://review.openstack.org/206376 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-etherpad_lite: Boilerplate beaker-rspec files https://review.openstack.org/206377 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-exim: Boilerplate beaker-rspec files https://review.openstack.org/206378 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-gerrit: Boilerplate beaker-rspec files https://review.openstack.org/206379 | 00:25 |
| *** marcusvrn has quit IRC | 00:25 | |
| openstackgerrit | Merged openstack-infra/puppet-gerritbot: Boilerplate beaker-rspec files https://review.openstack.org/206380 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-github: Boilerplate beaker-rspec files https://review.openstack.org/206381 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-graphite: Boilerplate beaker-rspec files https://review.openstack.org/206382 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-haveged: Boilerplate beaker-rspec files https://review.openstack.org/206383 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-httpd: Boilerplate beaker-rspec files https://review.openstack.org/206384 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-iptables: Boilerplate beaker-rspec files https://review.openstack.org/206385 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-jeepyb: Boilerplate beaker-rspec files https://review.openstack.org/206386 | 00:25 |
| harlowja | mordred i can talk with scott about it, i'm not sure he'd be ok with that though :-/ | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-jenkins: Boilerplate beaker-rspec files https://review.openstack.org/206387 | 00:25 |
| openstackgerrit | Merged openstack-infra/puppet-kerberos: Boilerplate beaker-rspec files https://review.openstack.org/206388 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-kibana: Boilerplate beaker-rspec files https://review.openstack.org/206389 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-lodgeit: Boilerplate beaker-rspec files https://review.openstack.org/206390 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor: Boilerplate beaker-rspec files https://review.openstack.org/206391 | 00:26 |
| *** aukhan has joined #openstack-infra | 00:26 | |
| harlowja | i agree with u, but i lack the power to make that happen :-/ | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-logrotate: Boilerplate beaker-rspec files https://review.openstack.org/206392 | 00:26 |
| *** aukhan has quit IRC | 00:26 | |
| openstackgerrit | Merged openstack-infra/puppet-logstash: Boilerplate beaker-rspec files https://review.openstack.org/206393 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-mailman: Boilerplate beaker-rspec files https://review.openstack.org/206394 | 00:26 |
| mordred | I hear that. maybe the 0.7 branch should just be ignored for now then? | 00:26 |
| *** tiswanso has quit IRC | 00:26 | |
| *** AzherKhan has quit IRC | 00:26 | |
| openstackgerrit | Merged openstack-infra/puppet-mediawiki: Boilerplate beaker-rspec files https://review.openstack.org/206395 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-meetbot: Boilerplate beaker-rspec files https://review.openstack.org/206396 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-mysql_backup: Boilerplate beaker-rspec files https://review.openstack.org/206397 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-nodepool: Boilerplate beaker-rspec files https://review.openstack.org/206398 | 00:26 |
| *** AzherKhan has joined #openstack-infra | 00:26 | |
| openstackgerrit | Merged openstack-infra/puppet-openafs: Boilerplate beaker-rspec files https://review.openstack.org/206399 | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-openstackid: Boilerplate beaker-rspec files https://review.openstack.org/206400 | 00:26 |
| harlowja | mordred possibly | 00:26 |
| openstackgerrit | Merged openstack-infra/puppet-os_client_config: Boilerplate beaker-rspec files https://review.openstack.org/206401 | 00:27 |
| openstackgerrit | Merged openstack-infra/puppet-packagekit: Boilerplate beaker-rspec files https://review.openstack.org/206402 | 00:27 |
| openstackgerrit | Merged openstack-infra/puppet-pgsql_backup: Boilerplate beaker-rspec files https://review.openstack.org/206403 | 00:27 |
| openstackgerrit | Merged openstack-infra/puppet-phabricator: Boilerplate beaker-rspec files https://review.openstack.org/206404 | 00:27 |
| openstackgerrit | Merged openstack-infra/puppet-pip: Boilerplate beaker-rspec files https://review.openstack.org/206405 | 00:27 |
| openstackgerrit | Merged openstack-infra/puppet-planet: Boilerplate beaker-rspec files https://review.openstack.org/206406 | 00:27 |
| *** soren has quit IRC | 00:27 | |
| openstackgerrit | Merged openstack-infra/puppet-project_config: Boilerplate beaker-rspec files https://review.openstack.org/206407 | 00:28 |
| harlowja | mordred i | 00:29 |
| openstackgerrit | Merged openstack-infra/puppet-puppet: Boilerplate beaker-rspec files https://review.openstack.org/206408 | 00:29 |
| *** taron has quit IRC | 00:29 | |
| harlowja | *i'll drag scott in here and we can ask, unsure still if he really would be up to stopping bzr branch development | 00:29 |
| harlowja | *or accepting stuff there | 00:29 |
| harlowja | i'm fine with it, buttttt, ya, well ya u know, ha | 00:30 |
| *** sabeen3 has quit IRC | 00:30 | |
| *** Sukhdev_ has joined #openstack-infra | 00:30 | |
| harlowja | if not, maybe should just delete 0.7.x then | 00:30 |
| harlowja | on stackforge, and leave it be | 00:31 |
| *** salv-orlando has quit IRC | 00:31 | |
| openstackgerrit | Merged openstack-infra/puppet-redis: Boilerplate beaker-rspec files https://review.openstack.org/206409 | 00:31 |
| openstackgerrit | Merged openstack-infra/puppet-releasestatus: Boilerplate beaker-rspec files https://review.openstack.org/206410 | 00:31 |
| openstackgerrit | Merged openstack-infra/puppet-reviewday: Boilerplate beaker-rspec files https://review.openstack.org/206411 | 00:31 |
| *** laughterwym has joined #openstack-infra | 00:32 | |
| openstackgerrit | Merged openstack-infra/puppet-simpleproxy: Boilerplate beaker-rspec files https://review.openstack.org/206412 | 00:33 |
| openstackgerrit | Merged openstack-infra/puppet-snmpd: Boilerplate beaker-rspec files https://review.openstack.org/206413 | 00:33 |
| *** gildub has joined #openstack-infra | 00:35 | |
| *** dannywilson has quit IRC | 00:36 | |
| *** Sukhdev_ has quit IRC | 00:37 | |
| *** tjones has quit IRC | 00:38 | |
| openstackgerrit | Merged openstack-infra/puppet-ssh: Boilerplate beaker-rspec files https://review.openstack.org/206414 | 00:39 |
| openstackgerrit | Merged openstack-infra/puppet-ssl_cert_check: Boilerplate beaker-rspec files https://review.openstack.org/206415 | 00:39 |
| openstackgerrit | Merged openstack-infra/puppet-statusbot: Boilerplate beaker-rspec files https://review.openstack.org/206416 | 00:39 |
| *** banix has quit IRC | 00:39 | |
| openstackgerrit | Merged openstack-infra/puppet-storyboard: Boilerplate beaker-rspec files https://review.openstack.org/206417 | 00:39 |
| openstackgerrit | Merged openstack-infra/puppet-subunit2sql: Boilerplate beaker-rspec files https://review.openstack.org/206418 | 00:41 |
| openstackgerrit | Merged openstack-infra/puppet-sudoers: Boilerplate beaker-rspec files https://review.openstack.org/206419 | 00:43 |
| *** nightfly has quit IRC | 00:43 | |
| *** bhuvan has joined #openstack-infra | 00:45 | |
| openstackgerrit | Merged openstack-infra/puppet-tmpreaper: Boilerplate beaker-rspec files https://review.openstack.org/206420 | 00:46 |
| *** bhuvan has quit IRC | 00:50 | |
| openstackgerrit | Merged openstack-infra/puppet-ulimit: Boilerplate beaker-rspec files https://review.openstack.org/206421 | 00:52 |
| openstackgerrit | Merged openstack-infra/puppet-unattended_upgrades: Boilerplate beaker-rspec files https://review.openstack.org/206422 | 00:52 |
| openstackgerrit | Merged openstack-infra/puppet-unbound: Boilerplate beaker-rspec files https://review.openstack.org/206423 | 00:52 |
| openstackgerrit | Merged openstack-infra/puppet-user: Boilerplate beaker-rspec files https://review.openstack.org/206424 | 00:52 |
| openstackgerrit | Merged openstack-infra/puppet-yum: Boilerplate beaker-rspec files https://review.openstack.org/206427 | 00:52 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be smarter finding private IP https://review.openstack.org/207157 | 00:52 |
| Shrews | mordred: fixed pep8 error because i suck ^^^ | 00:53 |
| *** dimtruck is now known as zz_dimtruck | 00:53 | |
| *** otter768 has joined #openstack-infra | 00:54 | |
| *** geoffarnold has quit IRC | 00:54 | |
| openstackgerrit | Merged openstack-infra/puppet-zanata: Boilerplate beaker-rspec files https://review.openstack.org/206428 | 00:55 |
| *** devkulkarni has joined #openstack-infra | 00:55 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Boilerplate beaker-rspec files https://review.openstack.org/206429 | 00:56 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/project-config: Add repo for apps-catalog-ui https://review.openstack.org/207253 | 00:57 |
| *** amotoki has joined #openstack-infra | 00:58 | |
| *** pal has joined #openstack-infra | 00:58 | |
| *** jasondotstar has joined #openstack-infra | 00:58 | |
| *** mriedem has joined #openstack-infra | 00:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental scale testing job for GBP https://review.openstack.org/206680 | 01:04 |
| jroll | Shrews: if you actually sucked you wouldn't fix your errors | 01:05 |
| *** tjones has joined #openstack-infra | 01:08 | |
| *** banix has joined #openstack-infra | 01:09 | |
| *** davideagnello has quit IRC | 01:12 | |
| openstackgerrit | Merged openstack-infra/project-config: Run full multinode tests against new dib images https://review.openstack.org/207187 | 01:13 |
| *** pal_ has joined #openstack-infra | 01:14 | |
| *** pal has quit IRC | 01:16 | |
| *** bknudson has joined #openstack-infra | 01:17 | |
| openstackgerrit | Sachi King proposed openstack-infra/project-config: Add constraints jobs to Jenkins https://review.openstack.org/207262 | 01:20 |
| *** banix has quit IRC | 01:22 | |
| *** Hunner has quit IRC | 01:23 | |
| *** glauco has quit IRC | 01:23 | |
| *** devkulkarni has quit IRC | 01:24 | |
| *** abitha has quit IRC | 01:24 | |
| *** madong_ is now known as larainema | 01:26 | |
| *** chlong has joined #openstack-infra | 01:29 | |
| *** jasondotstar has quit IRC | 01:35 | |
| *** esmute_ has quit IRC | 01:37 | |
| *** weshay has joined #openstack-infra | 01:37 | |
| openstackgerrit | Sachi King proposed openstack-infra/project-config: Add constraints jobs to Nova experemental queue https://review.openstack.org/207268 | 01:39 |
| *** bhunter71 has quit IRC | 01:40 | |
| Nakato | lifeless: ^ | 01:40 |
| *** shashank_hegde has quit IRC | 01:43 | |
| *** shashank_hegde has joined #openstack-infra | 01:44 | |
| *** banix has joined #openstack-infra | 01:45 | |
| *** MarkAtwood has joined #openstack-infra | 01:47 | |
| *** AzherKhan has quit IRC | 01:47 | |
| *** dan has quit IRC | 01:50 | |
| *** kun_huang has joined #openstack-infra | 01:51 | |
| *** johnbelamaric has joined #openstack-infra | 01:59 | |
| *** baoli has joined #openstack-infra | 02:02 | |
| *** Kennan has quit IRC | 02:02 | |
| *** Kennan has joined #openstack-infra | 02:02 | |
| *** yamamoto has joined #openstack-infra | 02:03 | |
| *** annegentle has quit IRC | 02:06 | |
| *** shashank_hegde has quit IRC | 02:07 | |
| *** skylerberg has quit IRC | 02:08 | |
| *** esmute_ has joined #openstack-infra | 02:09 | |
| *** esmute_ has quit IRC | 02:09 | |
| *** glauco has joined #openstack-infra | 02:11 | |
| *** esmute_ has joined #openstack-infra | 02:11 | |
| *** glauco has quit IRC | 02:15 | |
| *** esmute_ has quit IRC | 02:16 | |
| *** johnbelamaric has quit IRC | 02:16 | |
| *** abitha has joined #openstack-infra | 02:16 | |
| *** banix has quit IRC | 02:17 | |
| *** dims has quit IRC | 02:19 | |
| *** unicell has quit IRC | 02:19 | |
| *** dims has joined #openstack-infra | 02:19 | |
| *** tiswanso has joined #openstack-infra | 02:19 | |
| *** rlandy has joined #openstack-infra | 02:21 | |
| *** tiswanso has quit IRC | 02:22 | |
| *** tiswanso has joined #openstack-infra | 02:22 | |
| *** akshai has quit IRC | 02:23 | |
| *** akshai has joined #openstack-infra | 02:23 | |
| *** Somay has joined #openstack-infra | 02:23 | |
| *** akshai has quit IRC | 02:24 | |
| *** BharatK has quit IRC | 02:26 | |
| *** sergmelikyan has quit IRC | 02:28 | |
| *** jswarren has quit IRC | 02:28 | |
| *** mriedem1 has joined #openstack-infra | 02:29 | |
| *** jasondotstar has joined #openstack-infra | 02:31 | |
| *** tjones has quit IRC | 02:31 | |
| *** mriedem has quit IRC | 02:31 | |
| *** esmute_ has joined #openstack-infra | 02:31 | |
| *** esmute_ has quit IRC | 02:32 | |
| *** gildub has quit IRC | 02:33 | |
| *** esmute_ has joined #openstack-infra | 02:37 | |
| *** rossella_s has quit IRC | 02:37 | |
| *** rossella_s has joined #openstack-infra | 02:38 | |
| *** earlephilhower_ has joined #openstack-infra | 02:39 | |
| *** davideagnello has joined #openstack-infra | 02:39 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support for monitoring files using FSTrigger Plugin https://review.openstack.org/207275 | 02:39 |
| *** earlephilhower has quit IRC | 02:41 | |
| *** jasondotstar has quit IRC | 02:41 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support for monitoring files using FSTrigger Plugin https://review.openstack.org/204125 | 02:41 |
| *** ajmiller has joined #openstack-infra | 02:41 | |
| *** BharatK has joined #openstack-infra | 02:43 | |
| *** pal_ has quit IRC | 02:43 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support for monitoring files using FSTrigger Plugin https://review.openstack.org/203364 | 02:43 |
| *** bknudson has quit IRC | 02:43 | |
| *** nelsnelson has quit IRC | 02:45 | |
| *** changbl has quit IRC | 02:47 | |
| kun_huang | gate is stuck again | 02:49 |
| *** annegentle has joined #openstack-infra | 02:50 | |
| *** sergmelikyan has joined #openstack-infra | 02:51 | |
| *** esmute_ has quit IRC | 02:51 | |
| *** apmelton has quit IRC | 02:51 | |
| *** aukhan has joined #openstack-infra | 02:53 | |
| *** nelsnelson has joined #openstack-infra | 02:53 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Adding support to monitor folders using FS trigger plugin https://review.openstack.org/204125 | 02:53 |
| *** skylerberg has joined #openstack-infra | 02:53 | |
| *** apmelton has joined #openstack-infra | 02:54 | |
| openstackgerrit | liusheng proposed openstack/requirements: Bump gabbi to version 1.1.4 https://review.openstack.org/201953 | 02:54 |
| *** esmute_ has joined #openstack-infra | 02:54 | |
| openstackgerrit | liusheng proposed openstack/requirements: Bump tooz to version 1.19.0 https://review.openstack.org/205730 | 02:56 |
| *** dims has quit IRC | 02:57 | |
| *** ajmiller has quit IRC | 02:58 | |
| *** ajmiller has joined #openstack-infra | 02:58 | |
| *** esmute_ has quit IRC | 03:04 | |
| *** earlephilhower_ is now known as earlephilhower | 03:04 | |
| *** esmute_ has joined #openstack-infra | 03:04 | |
| *** sdake has joined #openstack-infra | 03:05 | |
| *** changbl has joined #openstack-infra | 03:06 | |
| *** weshay has quit IRC | 03:08 | |
| *** maishsk has joined #openstack-infra | 03:12 | |
| *** baoli has quit IRC | 03:13 | |
| *** woodster_ has quit IRC | 03:14 | |
| *** tjones has joined #openstack-infra | 03:15 | |
| *** rfolco has quit IRC | 03:17 | |
| *** kevinbenton has quit IRC | 03:18 | |
| *** annegentle has quit IRC | 03:20 | |
| *** markvoelker_ has quit IRC | 03:20 | |
| *** kevinbenton has joined #openstack-infra | 03:20 | |
| *** markvoelker_ has joined #openstack-infra | 03:22 | |
| *** tiswanso has quit IRC | 03:25 | |
| *** annegentle has joined #openstack-infra | 03:26 | |
| *** annegentle has quit IRC | 03:26 | |
| *** devkulkarni has joined #openstack-infra | 03:28 | |
| openstackgerrit | Somay Jain proposed openstack-infra/jenkins-job-builder: Support for CVS SCM using CVS plugin https://review.openstack.org/203685 | 03:29 |
| *** dtantsur|afk has quit IRC | 03:31 | |
| *** dtantsur has joined #openstack-infra | 03:31 | |
| *** julim has quit IRC | 03:32 | |
| *** laughterwym_ has joined #openstack-infra | 03:36 | |
| *** laughterwym has quit IRC | 03:36 | |
| *** sdake has quit IRC | 03:36 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-haveged: Add Gemfile and puppet 4 checks https://review.openstack.org/207283 | 03:38 |
| *** annegentle has joined #openstack-infra | 03:40 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-hound: Add Gemfile and puppet 4 checks https://review.openstack.org/207284 | 03:41 |
| ianw | jhesketh: whatever you did to fix zuul, you might need to do it again :) | 03:45 |
| *** maishsk has quit IRC | 03:45 | |
| *** jamielennox|away is now known as jamielennox | 03:47 | |
| *** otter768 has quit IRC | 03:50 | |
| *** esker has joined #openstack-infra | 03:53 | |
| *** kevinbenton has quit IRC | 03:56 | |
| *** coolsvap|away is now known as coolsvap | 03:59 | |
| *** rlandy has quit IRC | 03:59 | |
| *** kevinbenton has joined #openstack-infra | 04:01 | |
| *** aukhan has quit IRC | 04:02 | |
| *** aluria has quit IRC | 04:05 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: puppet: run copy_logs Rake task on beaker jobs https://review.openstack.org/206693 | 04:07 |
| *** aukhan has joined #openstack-infra | 04:07 | |
| *** Somay has quit IRC | 04:09 | |
| *** harlowja_at_home has joined #openstack-infra | 04:13 | |
| *** pal has joined #openstack-infra | 04:15 | |
| *** esmute_ has quit IRC | 04:15 | |
| *** btully has quit IRC | 04:16 | |
| *** sabeen1 has joined #openstack-infra | 04:19 | |
| *** salv-orlando has joined #openstack-infra | 04:21 | |
| *** yfried_ has joined #openstack-infra | 04:25 | |
| *** salv-orlando has quit IRC | 04:25 | |
| *** achanda has quit IRC | 04:26 | |
| *** serg_melikyan has joined #openstack-infra | 04:26 | |
| *** sergmelikyan has quit IRC | 04:27 | |
| *** fawadkhaliq has joined #openstack-infra | 04:27 | |
| EmilienM | ianw: maybe we can chat a bit before I go to sleep | 04:28 |
| *** dingyichen has quit IRC | 04:29 | |
| ianw | EmilienM: sure, I just don't think the log copying really belongs out in the puppet modules. i mean, we don't have devstack copying logs into place for picking up, we leave that to the infrastructure | 04:29 |
| *** dingyichen has joined #openstack-infra | 04:29 | |
| EmilienM | ianw: that's why we use the Rakefile | 04:29 |
| EmilienM | because each module isntalls a component, we don't test everything in the module | 04:29 |
| EmilienM | ie: puppet-keystone will test keystone so we only need keystone.log and apache logs eventually | 04:30 |
| EmilienM | so there is no generic (except copying /var/log/*) | 04:30 |
| ianw | EmilienM: right, that's fine, but if I'm using puppet-keystone, as a person who knows nothing about openstack CI, why do I want a Rakefile target to copy logs to $WORKSPACE? | 04:30 |
| ianw | it's just the type of thing that should be in a macro alongside the job definition | 04:31 |
| EmilienM | ianw: it won't | 04:31 |
| EmilienM | see https://review.openstack.org/199712 | 04:31 |
| EmilienM | we only copy logs if we explicitly run the Rake rask | 04:31 |
| EmilienM | and even if we run the Rake task, we check if $WORKSPACE is set (when running on a jenkins node) | 04:31 |
| ianw | right, but why would that be in puppet-keystone at all? | 04:32 |
| ianw | it's totally specific to the upstream CI environment | 04:32 |
| *** jlvillal has quit IRC | 04:32 | |
| EmilienM | ianw: what would you copy then from the generic CI ? | 04:32 |
| EmilienM | /var/log? | 04:32 |
| *** gsagie has joined #openstack-infra | 04:32 | |
| ianw | you can copy whatever you want, i'm just saying it should be in the job definition yaml files | 04:32 |
| ianw | like i think the way it started out | 04:33 |
| EmilienM | ianw: the job definition is standard | 04:33 |
| EmilienM | it runs a (standard) task to copy logs | 04:33 |
| gsagie | Hello everyone, i have a patch pending for irc-meetings change, if anyone could review it we hope it gets merged until Monday next week (which is our scheduled meeting :) https://review.openstack.org/#/c/206324/ | 04:33 |
| EmilienM | and it's up to the module to say which logs we want | 04:33 |
| EmilienM | it's different for every component | 04:33 |
| *** jamielennox is now known as jamielennox|away | 04:33 | |
| EmilienM | ie: keystone will need keystone.log & apache logs | 04:33 |
| ianw | EmilienM: there's no problems adding more steps. just those steps should be added as part of the job that runs puppet-keystone testing | 04:34 |
| crinkle | why not just copy all of /var/log? | 04:35 |
| EmilienM | ianw: if you look at the history of https://review.openstack.org/199712 - I started by copying the logs directly from the puppet module - and I got the feedback to use Rake for that. Using rake requires to run the task from jenkins job | 04:35 |
| EmilienM | crinkle: you here | 04:35 |
| *** devkulkarni has quit IRC | 04:35 | |
| EmilienM | I can copy whatever you like | 04:35 |
| EmilienM | I'm just hoping having logs for a while now, I don't really care how | 04:35 |
| EmilienM | crinkle: to me, it's not optimal to copy the whole /var/log, also it takes some place | 04:36 |
| ianw | EmilienM: it doesn't make sense to move all this openstack-ci specific logic into every single puppet-* project | 04:36 |
| crinkle | I like ianw's proposal to do it in jjb instead of maintaining individual rake tasks for each module | 04:36 |
| EmilienM | crinkle: you +2 my patch of copying only keystone.log, I'm confused now | 04:36 |
| crinkle | EmilienM: I did but I didn't know the infrastructure well enough to consider this option | 04:36 |
| ianw | for example, puppet-keystone wouldn't merge changes from hp, ibm, blah blah all of them to copy logs where their ci wanted them | 04:37 |
| EmilienM | cool, so I change the patch to copy /var/log? | 04:37 |
| ianw | (at least i hope not) | 04:37 |
| crinkle | ianw: do you have an example of a job that copies specific log files? | 04:37 |
| crinkle | how big is /var/log? is it unreasonable to copy the whole thing? | 04:37 |
| ianw | let me look; devstack-gate handles this for devstack | 04:37 |
| *** yamamoto has quit IRC | 04:38 | |
| *** serg_melikyan has quit IRC | 04:40 | |
| *** taron has joined #openstack-infra | 04:40 | |
| ianw | EmilienM: so the issue was the logs didn't get copied on failure right | 04:41 |
| ianw | when you just did a cp ... $WORKSPACE at the end | 04:41 |
| EmilienM | ianw: what issue? | 04:42 |
| EmilienM | I'm implementing a feature, there is no issue afik | 04:42 |
| ianw | well, that was what you were working around with the $? checking I think i reviewed? | 04:42 |
| EmilienM | in the Rakefile, I'm sanitizing the task run by checking $WORSPACE is set | 04:42 |
| EmilienM | there is no issue at all in this story | 04:43 |
| *** sergmelikyan has joined #openstack-infra | 04:43 | |
| *** annegentle has quit IRC | 04:44 | |
| EmilienM | ianw: if you feel comfortable, you can take over my patch | 04:45 |
| EmilienM | crinkle: in our use case, I don't think it's useful to cp /var/log/* - a lot of useless data | 04:45 |
| crinkle | how does tripleo do it? they have relevant logs http://logs.openstack.org/80/172580/17/check-tripleo/check-tripleo-ironic-overcloud-f21puppet-nonha/04bfc1e/logs/ | 04:47 |
| *** dingyichen has quit IRC | 04:47 | |
| *** kun_huang has left #openstack-infra | 04:47 | |
| *** pal has quit IRC | 04:47 | |
| ianw | EmilienM: i don't want to take over your patch. i'll leave a comment on what i think is the best way forward | 04:47 |
| *** deepakcs has joined #openstack-infra | 04:48 | |
| EmilienM | ianw: awesome, I'm leaving to bed on my side, 1am here | 04:49 |
| *** sergmelikyan has quit IRC | 04:50 | |
| *** yamamoto has joined #openstack-infra | 04:52 | |
| *** fawadkhaliq has quit IRC | 04:52 | |
| *** yfried_ has quit IRC | 04:53 | |
| *** tjones has quit IRC | 04:54 | |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/devstack-gate: Allow for git base override https://review.openstack.org/151034 | 04:55 |
| *** darrenc is now known as darrenc_afk | 04:56 | |
| *** oomichi has quit IRC | 05:00 | |
| *** davideagnello has quit IRC | 05:00 | |
| *** oomichi has joined #openstack-infra | 05:05 | |
| *** oomichi has quit IRC | 05:05 | |
| *** fawadkhaliq has joined #openstack-infra | 05:05 | |
| *** pal has joined #openstack-infra | 05:06 | |
| *** harlowja_at_home has quit IRC | 05:10 | |
| *** yamahata has joined #openstack-infra | 05:10 | |
| *** abitha has quit IRC | 05:11 | |
| *** geoffarnold has joined #openstack-infra | 05:12 | |
| *** Hunner has joined #openstack-infra | 05:14 | |
| ianw | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=23 <- doesn't look like disk usage | 05:15 |
| *** mrmartin has joined #openstack-infra | 05:15 | |
| *** unicell has joined #openstack-infra | 05:15 | |
| *** geoffarnold has quit IRC | 05:15 | |
| *** geoffarnold has joined #openstack-infra | 05:18 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: telemetry: add integration tests job https://review.openstack.org/207298 | 05:20 |
| *** amickus has joined #openstack-infra | 05:21 | |
| *** iwamoto has joined #openstack-infra | 05:22 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: telemetry: add integration tests job https://review.openstack.org/207298 | 05:23 |
| *** koolhead17 has joined #openstack-infra | 05:25 | |
| *** gsagie has left #openstack-infra | 05:26 | |
| *** achanda has joined #openstack-infra | 05:29 | |
| *** sabeen1 has quit IRC | 05:29 | |
| *** btully has joined #openstack-infra | 05:30 | |
| *** laughterwym__ has joined #openstack-infra | 05:32 | |
| *** sabeen1 has joined #openstack-infra | 05:32 | |
| *** alexsyip has joined #openstack-infra | 05:33 | |
| *** btully has quit IRC | 05:34 | |
| *** laughterwym_ has quit IRC | 05:35 | |
| *** tangchen has quit IRC | 05:35 | |
| *** alexsyip has quit IRC | 05:38 | |
| *** darrenc_afk is now known as darrenc | 05:40 | |
| *** dannywilson has joined #openstack-infra | 05:49 | |
| *** otter768 has joined #openstack-infra | 05:51 | |
| *** shashank_hegde has joined #openstack-infra | 05:54 | |
| *** erikwilson_ has quit IRC | 05:54 | |
| *** salv-orlando has joined #openstack-infra | 05:54 | |
| *** erikwilson_ has joined #openstack-infra | 05:55 | |
| *** salv-orlando has quit IRC | 05:55 | |
| *** otter768 has quit IRC | 05:55 | |
| *** jtomasek has quit IRC | 05:56 | |
| *** flepied1 has quit IRC | 05:56 | |
| *** kushal has quit IRC | 05:56 | |
| *** salv-orlando has joined #openstack-infra | 05:56 | |
| *** erikwilson_ has quit IRC | 05:59 | |
| *** Kennan2 has joined #openstack-infra | 06:03 | |
| *** Kennan has quit IRC | 06:03 | |
| *** BharatK has quit IRC | 06:04 | |
| *** dannywilson has quit IRC | 06:11 | |
| *** dannywilson has joined #openstack-infra | 06:12 | |
| *** esker has quit IRC | 06:13 | |
| *** spredzy|afk is now known as spredzy | 06:17 | |
| *** dingyichen has joined #openstack-infra | 06:18 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-httpd: Fix $httpd::vhost::proxy::servername parameter https://review.openstack.org/207310 | 06:18 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-httpd: Add Gemfile and puppet 4 checks https://review.openstack.org/207311 | 06:18 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rename .log to .log.txt in test-results publisher https://review.openstack.org/203099 | 06:19 |
| *** dingyichen has quit IRC | 06:19 | |
| *** AJaeger has joined #openstack-infra | 06:20 | |
| *** dingyichen has joined #openstack-infra | 06:20 | |
| AJaeger | Hi jhesketh, are you around? Could it be that Zuul is unhappy again? ;( | 06:20 |
| AJaeger | jhesketh, clarkb, fungi, pleia2, jeblair: Zuul is stuck again. Can either of you help, please? | 06:24 |
| *** MarkAtwood has quit IRC | 06:24 | |
| *** sabeen1 has quit IRC | 06:26 | |
| AJaeger | ttx, lifeless: Are you around to update the topic again? | 06:27 |
| *** MarkAtwood has joined #openstack-infra | 06:28 | |
| *** thingee_ is now known as thingee | 06:33 | |
| *** Somay has joined #openstack-infra | 06:35 | |
| *** skylerberg has quit IRC | 06:36 | |
| *** yfried_ has joined #openstack-infra | 06:36 | |
| *** rossella_s has quit IRC | 06:37 | |
| *** rossella_s has joined #openstack-infra | 06:38 | |
| *** oomichi has joined #openstack-infra | 06:40 | |
| *** yfried_ has quit IRC | 06:41 | |
| *** yfried__ has joined #openstack-infra | 06:41 | |
| *** vlaza has joined #openstack-infra | 06:43 | |
| *** david-lyle has quit IRC | 06:43 | |
| *** liam__ has joined #openstack-infra | 06:46 | |
| *** liam__ has quit IRC | 06:46 | |
| *** BharatK has joined #openstack-infra | 06:48 | |
| *** scheuran has joined #openstack-infra | 06:48 | |
| *** david-lyle has joined #openstack-infra | 06:49 | |
| *** mpavone has joined #openstack-infra | 06:50 | |
| *** kushal has joined #openstack-infra | 06:50 | |
| *** BharatK has quit IRC | 06:51 | |
| *** BharatK has joined #openstack-infra | 06:51 | |
| *** larainema has quit IRC | 06:51 | |
| *** _nadya_ has joined #openstack-infra | 06:52 | |
| *** vinay_yadhav has joined #openstack-infra | 06:52 | |
| *** gildub has joined #openstack-infra | 06:53 | |
| *** MarkAtwood has quit IRC | 06:55 | |
| *** larainema has joined #openstack-infra | 06:59 | |
| *** davideagnello has joined #openstack-infra | 07:01 | |
| *** ifarkas has joined #openstack-infra | 07:03 | |
| *** Ala has joined #openstack-infra | 07:04 | |
| *** davideagnello has quit IRC | 07:05 | |
| *** doude has joined #openstack-infra | 07:05 | |
| *** unicell has quit IRC | 07:08 | |
| *** unicell has joined #openstack-infra | 07:09 | |
| *** flepied has joined #openstack-infra | 07:10 | |
| *** salv-orlando has quit IRC | 07:15 | |
| *** _nadya_ has quit IRC | 07:16 | |
| *** _nadya_ has joined #openstack-infra | 07:16 | |
| *** markus_z has joined #openstack-infra | 07:16 | |
| *** vinay_yadhav has quit IRC | 07:17 | |
| *** dannywilson has quit IRC | 07:18 | |
| *** devvesa has joined #openstack-infra | 07:20 | |
| *** achanda has quit IRC | 07:20 | |
| *** _nadya_ has quit IRC | 07:20 | |
| *** achanda has joined #openstack-infra | 07:23 | |
| *** arxcruz has joined #openstack-infra | 07:23 | |
| *** unicell has quit IRC | 07:24 | |
| *** chlong has quit IRC | 07:25 | |
| *** soren has joined #openstack-infra | 07:26 | |
| *** jcoufal has joined #openstack-infra | 07:26 | |
| *** shashank_hegde has quit IRC | 07:26 | |
| *** fhubik has joined #openstack-infra | 07:27 | |
| *** unicell has joined #openstack-infra | 07:27 | |
| *** ParsectiX has joined #openstack-infra | 07:27 | |
| *** hichihara has quit IRC | 07:32 | |
| *** BharatK_ has joined #openstack-infra | 07:33 | |
| *** shashank_hegde has joined #openstack-infra | 07:33 | |
| *** cdent has joined #openstack-infra | 07:34 | |
| *** BharatK has quit IRC | 07:35 | |
| *** glauco_ has joined #openstack-infra | 07:36 | |
| ttx | AJaeger: yes | 07:36 |
| ttx | same | 07:36 |
| ttx | ? | 07:36 |
| AJaeger | ttx, yeah ;( | 07:37 |
| ttx | #status Our CI system is broken again today, jobs are not getting processed at all. | 07:38 |
| openstackstatus | ttx: unknown command | 07:38 |
| ttx | #status alert Our CI system is broken again today, jobs are not getting processed at all. | 07:38 |
| openstackstatus | ttx: sending alert | 07:38 |
| AJaeger | ttx, Zuul is stuck completely again and no infra root around to help so far. | 07:38 |
| AJaeger | thanks, ttx! | 07:38 |
| StevenK | I think jhesketh is probably on his way to Brisbane | 07:39 |
| jhesketh | just got into Brisbane, I'll take a look | 07:39 |
| StevenK | jhesketh: \o/ | 07:39 |
| AJaeger | Thanks, jhesketh and StevenK ! | 07:40 |
| AJaeger | jhesketh: any explanation why Zuul is only stuck when the US is asleep? ;) | 07:40 |
| -openstackstatus- NOTICE: Our CI system is broken again today, jobs are not getting processed at all. | 07:40 | |
| *** ChanServ changes topic to "Our CI system is broken again today, jobs are not getting processed at all." | 07:40 | |
| *** glauco_ has quit IRC | 07:40 | |
| *** mpaolino has joined #openstack-infra | 07:41 | |
| jhesketh | so yes, same problem as earlier in the week and last week | 07:41 |
| jhesketh | I wonder if we should disable zuul reloads | 07:41 |
| jhesketh | or not merge any layout updates until we've tracked down the bug(s) | 07:42 |
| *** kun_huang has joined #openstack-infra | 07:42 | |
| jhesketh | wow, the pipelines are huge, this isn't going to be pretty | 07:42 |
| *** shashank_hegde has quit IRC | 07:42 | |
| kun_huang | welcome back | 07:42 |
| AJaeger | jhesketh: indeed ;( | 07:42 |
| jhesketh | the only bandaid I know of is to restart the services which will put us a very long way behind | 07:43 |
| jhesketh | and that doesn't fix the root cause that we havne't solved yet | 07:43 |
| openstackstatus | ttx: finished sending alert | 07:43 |
| ttx | jhesketh: still a better option than doing nothing and letting changes accumulate, no ? | 07:43 |
| AJaeger | jeblair wanted to add logging to signals so that we sees what'S broken. Right now we miss logging output (unless he made the change in the last 12 hours) for this situation... | 07:44 |
| jhesketh | ttx: quite likely | 07:44 |
| ttx | agree that until it's tracked down zuul should probably only reload while infra-core is around to fix it | 07:44 |
| jhesketh | AJaeger: I believe he was going to add those manually and restart zuul | 07:44 |
| jhesketh | I doubt he's done that | 07:44 |
| jhesketh | ttx: I'm not convinced it's a reload that's doing it, but that's my guess | 07:45 |
| ttx | it's certainly something that happens at the same hour every day | 07:46 |
| jhesketh | it looks like the current one has been stuck 15 hours | 07:46 |
| *** amickus has quit IRC | 07:46 | |
| AJaeger | ttx: A bit earlier this day - ianw noticed it at 7am CEST | 07:46 |
| *** kun_huang has left #openstack-infra | 07:46 | |
| AJaeger | 15 hours? Oops ;( | 07:46 |
| *** ihrachyshka has joined #openstack-infra | 07:46 | |
| jhesketh | well that's how long the blocking change has been queued for | 07:46 |
| ttx | jhesketh: not 15 hours | 07:46 |
| jhesketh | so probably 4 or so less than that | 07:46 |
| jhesketh | depends how far behind the queue was when it entered | 07:46 |
| ttx | jhesketh: the queue was backed up a few hours already when the freeze happened | 07:46 |
| *** geoffarnold has quit IRC | 07:47 | |
| ttx | the graphs at the bottom are pretty clear | 07:47 |
| jhesketh | right and there were nodepool problems or something I missed overnight | 07:47 |
| ttx | more like a 3am UTC thing | 07:47 |
| *** geoffarnold has joined #openstack-infra | 07:47 | |
| ttx | or 2am | 07:47 |
| jhesketh | #status alert CI system is broken and very far behind. Please do not approve any changes for a while. | 07:48 |
| openstackstatus | jhesketh: sending alert | 07:48 |
| jhesketh | okay I'm going to restart things as it's probably the best option we have | 07:48 |
| *** gildub has quit IRC | 07:49 | |
| -openstackstatus- NOTICE: CI system is broken and very far behind. Please do not approve any changes for a while. | 07:50 | |
| *** ChanServ changes topic to "CI system is broken and very far behind. Please do not approve any changes for a while." | 07:50 | |
| *** otter768 has joined #openstack-infra | 07:52 | |
| openstackstatus | jhesketh: finished sending alert | 07:52 |
| *** erikwilson_ has joined #openstack-infra | 07:55 | |
| *** achanda has quit IRC | 07:56 | |
| *** abregman has joined #openstack-infra | 07:56 | |
| *** otter768 has quit IRC | 07:56 | |
| *** jamielennox|away is now known as jamielennox | 07:57 | |
| lifeless | AJaeger: hi | 07:57 |
| lifeless | AJaeger: whatsup? I just got off of a plane | 07:58 |
| AJaeger | lifeless: jhesketh and ttx have helped already, thanks! Enjoy Brisbane! | 07:58 |
| lifeless | AJaeger: ack, shall do | 07:58 |
| *** achanda has joined #openstack-infra | 07:58 | |
| *** erikwilson_ has quit IRC | 08:00 | |
| greghaynes | jhesketh: :( so disk filling wasnt the issue? | 08:05 |
| jhesketh | greghaynes: no, it's a different issue that has been troubling us for about a week | 08:05 |
| jhesketh | jeblair and clarkb were working on debugging it, I haven't had a chance unfortunately | 08:05 |
| lifeless | ok, 2 stops to get out. afk again. Also, first talk nearly finished, Agenda: more lkhole, more talks, more fun. | 08:06 |
| *** achanda has quit IRC | 08:07 | |
| marekd | AJaeger: Hi, Do i remember correctly that you were also responsible for translations in OpenStack ? | 08:07 |
| AJaeger | marekd: I'm involved in some parts, so is StevenK - and many on #openstack-i18n channel. What's your question? | 08:08 |
| marekd | https://review.openstack.org/#/c/206987/2/keystone/auth/plugins/mapped.py PEP8 is complaining about my one would have thought simple cange saying "Empty localization string". I have no doubt what's wrong . | 08:09 |
| AJaeger | marekd: let me check | 08:09 |
| marekd | AJaeger: thanks. | 08:09 |
| AJaeger | marekd: you can't do it this way, let me comment... | 08:09 |
| *** markus_z has quit IRC | 08:09 | |
| AJaeger | marekd: basically, it's _("real string") and not _(variable) | 08:10 |
| marekd | AJaeger: I tried doing msg = _(("multi" "line" "string") | 08:10 |
| AJaeger | marekd: that should have worked... | 08:11 |
| marekd | AJaeger: ok, i thin i shouldn't make it a "tuple" | 08:11 |
| *** abitha has joined #openstack-infra | 08:11 | |
| marekd | like i would make with standard multiline string. | 08:12 |
| marekd | anyway, you pointed me to a right direction! | 08:12 |
| marekd | thanks! | 08:12 |
| *** vinay_yadhav has joined #openstack-infra | 08:12 | |
| AJaeger | right, it should be _("multi" "line" "string") | 08:12 |
| marekd | AJaeger: yes! Thanks! | 08:13 |
| AJaeger | marekd: you're welcome | 08:13 |
| AJaeger | jhesketh: zuul seems to be happy again, thanks! | 08:15 |
| * AJaeger won't approve anything right now ;) | 08:15 | |
| jhesketh | AJaeger: yep, just re-queuing the worl | 08:15 |
| jhesketh | *world | 08:16 |
| *** abitha has quit IRC | 08:16 | |
| *** vinay_yadhav has quit IRC | 08:16 | |
| *** vinay_yadhav has joined #openstack-infra | 08:17 | |
| *** hashar has joined #openstack-infra | 08:17 | |
| *** fhubik is now known as fhubik_afk | 08:17 | |
| SergK | Hello, Could you please add me to fuel-mirror-core group, since my request https://review.openstack.org/#/c/205963/ was successfully merged. Thank you in advance | 08:18 |
| *** sobersabre has quit IRC | 08:18 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-gerrit: New logrotate class in gerrit https://review.openstack.org/201312 | 08:20 |
| *** zeih has joined #openstack-infra | 08:22 | |
| *** kiran-r has joined #openstack-infra | 08:25 | |
| *** jistr has joined #openstack-infra | 08:25 | |
| *** fhubik_afk is now known as fhubik | 08:25 | |
| AJaeger | thanks, jhesketh. Please remember to update the IRC bot, I guess we should ask for requing of some of the changes again | 08:26 |
| jhesketh | AJaeger: yep, will do once the queue is up to date | 08:27 |
| openstackgerrit | Merged openstack-infra/project-config: Make the Heat Convergence Gate job non voting https://review.openstack.org/206888 | 08:27 |
| jhesketh | heh, requeuing jobs seem to have included reloads to zuul ^ | 08:28 |
| jhesketh | I wonder if that'll break things again :-( | 08:28 |
| *** laughterwym__ has quit IRC | 08:28 | |
| openstackgerrit | Doron Chen proposed openstack-infra/project-config: adding a new stackforge/storlets project https://review.openstack.org/203418 | 08:31 |
| *** zeih has quit IRC | 08:32 | |
| openstackgerrit | Doron Chen proposed openstack-infra/project-config: adding a new openstack/storlets project https://review.openstack.org/203418 | 08:32 |
| *** enikanorov has joined #openstack-infra | 08:33 | |
| *** afazekas has joined #openstack-infra | 08:33 | |
| *** BobBall_AWOL is now known as BobBaool | 08:36 | |
| *** BobBaool is now known as BobBall | 08:36 | |
| *** salv-orlando has joined #openstack-infra | 08:37 | |
| *** mpaolino has quit IRC | 08:38 | |
| *** ZZelle has joined #openstack-infra | 08:38 | |
| *** k4n0 has joined #openstack-infra | 08:38 | |
| *** fhubik is now known as fhubik_afk | 08:39 | |
| *** mpaolino has joined #openstack-infra | 08:39 | |
| *** hashar_ has joined #openstack-infra | 08:40 | |
| *** vinay_yadhav has quit IRC | 08:41 | |
| *** vinayyadhav has joined #openstack-infra | 08:41 | |
| *** hashar has quit IRC | 08:42 | |
| *** gildub has joined #openstack-infra | 08:42 | |
| *** jamespage_ has joined #openstack-infra | 08:48 | |
| *** dtantsur is now known as dtantsur|brb | 08:48 | |
| *** salv-orlando has quit IRC | 08:49 | |
| *** sobersabre has joined #openstack-infra | 08:49 | |
| *** fawadkhaliq has quit IRC | 08:53 | |
| *** jamespage_ has quit IRC | 08:54 | |
| *** iwamoto has quit IRC | 08:54 | |
| *** jtomasek has joined #openstack-infra | 08:56 | |
| *** fhubik_afk is now known as fhubik | 08:56 | |
| jhesketh | #status ok CI is back online but has a huge backlog. Please be patient and if possible delay approving changes until it has caught up. | 08:57 |
| openstackstatus | jhesketh: sending ok | 08:57 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/" | 08:59 | |
| -openstackstatus- NOTICE: CI is back online but has a huge backlog. Please be patient and if possible delay approving changes until it has caught up. | 08:59 | |
| *** jlanoux has joined #openstack-infra | 09:01 | |
| openstackstatus | jhesketh: finished sending ok | 09:01 |
| *** jamespage_ has joined #openstack-infra | 09:02 | |
| *** MarkAtwood has joined #openstack-infra | 09:02 | |
| *** sshnaidm has joined #openstack-infra | 09:03 | |
| *** e0ne has joined #openstack-infra | 09:03 | |
| *** naohirot has joined #openstack-infra | 09:03 | |
| *** pelix has joined #openstack-infra | 09:06 | |
| *** chlong has joined #openstack-infra | 09:07 | |
| *** MarkAtwood has quit IRC | 09:07 | |
| *** achanda has joined #openstack-infra | 09:08 | |
| *** jtomasek has quit IRC | 09:09 | |
| *** jtomasek has joined #openstack-infra | 09:10 | |
| *** achanda has quit IRC | 09:13 | |
| *** hashar_ is now known as hashar | 09:13 | |
| *** vinay_yadhav has joined #openstack-infra | 09:14 | |
| *** vinayyadhav has quit IRC | 09:14 | |
| *** jtomasek has quit IRC | 09:15 | |
| *** abregman is now known as abregman|PTO | 09:15 | |
| *** fawadkhaliq has joined #openstack-infra | 09:25 | |
| *** jamielennox is now known as jamielennox|away | 09:25 | |
| *** fawadkhaliq has quit IRC | 09:26 | |
| *** fawadkhaliq has joined #openstack-infra | 09:26 | |
| *** jtomasek has joined #openstack-infra | 09:26 | |
| *** oomichi has quit IRC | 09:27 | |
| *** vinay_yadhav has quit IRC | 09:29 | |
| *** vinay_yadhav has joined #openstack-infra | 09:29 | |
| *** ig0r_ has joined #openstack-infra | 09:31 | |
| *** jamespage_ has quit IRC | 09:35 | |
| *** jcoufal has quit IRC | 09:35 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add Kuryr weekly IRC meeting https://review.openstack.org/206324 | 09:37 |
| *** salv-orlando has joined #openstack-infra | 09:38 | |
| *** AJaeger has quit IRC | 09:39 | |
| *** rick__ has joined #openstack-infra | 09:44 | |
| rick__ | anteaya: HI Anteaya! | 09:44 |
| *** _nadya_ has joined #openstack-infra | 09:45 | |
| rick__ | anteaya: My account "prophetstor-ci" is disabled, How can I enable it. | 09:46 |
| *** pblaho has joined #openstack-infra | 09:46 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Update Searchlight Team Meeting https://review.openstack.org/206875 | 09:49 |
| *** dims has joined #openstack-infra | 09:50 | |
| openstackgerrit | Merged openstack-infra/release-tools: only release one version at a time https://review.openstack.org/203772 | 09:51 |
| *** jcoufal has joined #openstack-infra | 09:52 | |
| openstackgerrit | Merged openstack-infra/release-tools: Ironic participates in stable point releases starting with Kilo https://review.openstack.org/203774 | 09:52 |
| *** otter768 has joined #openstack-infra | 09:53 | |
| *** naohirot has quit IRC | 09:54 | |
| *** coolsvap is now known as coolsvap|away | 09:55 | |
| *** neiljerr` has joined #openstack-infra | 09:55 | |
| neiljerr` | Good morning! Is this a good time to ask sthg related to new project creation? | 09:57 |
| *** marzif_ has joined #openstack-infra | 09:57 | |
| *** otter768 has quit IRC | 09:57 | |
| neiljerr` | Actually let's pend that, I need to pop elsewhere for 30 minutes or so... Will ask again a bit later! | 09:59 |
| *** yamamoto has quit IRC | 10:00 | |
| *** alexpilotti has joined #openstack-infra | 10:01 | |
| *** romcheg has joined #openstack-infra | 10:01 | |
| romcheg | Hi folks! | 10:01 |
| romcheg | This patch was approved yesterday but still in not merged https://review.openstack.org/#/c/206106/ | 10:01 |
| romcheg | I cannot find in in any of Zuul's queues | 10:01 |
| *** yfried__ is now known as yfried|afk | 10:02 | |
| *** jcoufal has quit IRC | 10:02 | |
| *** vinay_yadhav has quit IRC | 10:05 | |
| mika | https://review.openstack.org/#/c/207239/ - hm, i don't understand why the CI tests didn't run for my review yet, any hints? | 10:05 |
| *** abregman|PTO has quit IRC | 10:06 | |
| greghaynes | romcheg: things blew up and were restarted - you can recheck or if its not critical wait a bit and then recheck while we churn through backlog | 10:06 |
| romcheg | greghaynes: thanks! | 10:07 |
| *** yfried|afk is now known as yfried__ | 10:07 | |
| greghaynes | romcheg: actually, you may have wo un-approve then re-approve | 10:07 |
| *** romcheg has quit IRC | 10:09 | |
| *** rick__ has quit IRC | 10:11 | |
| *** HeOS has joined #openstack-infra | 10:11 | |
| *** jcoufal has joined #openstack-infra | 10:12 | |
| *** dtantsur|brb is now known as dtantsur | 10:12 | |
| *** jasondotstar has joined #openstack-infra | 10:15 | |
| *** romcheg has joined #openstack-infra | 10:22 | |
| *** yamahata has quit IRC | 10:30 | |
| *** devvesa has quit IRC | 10:30 | |
| *** devvesa has joined #openstack-infra | 10:33 | |
| *** yamamoto has joined #openstack-infra | 10:35 | |
| *** claudiub has joined #openstack-infra | 10:35 | |
| *** pal has quit IRC | 10:37 | |
| *** yamamoto has quit IRC | 10:39 | |
| *** fhubik is now known as fhubik_afk | 10:40 | |
| *** hashar has quit IRC | 10:42 | |
| *** yamamoto has joined #openstack-infra | 10:43 | |
| *** fhubik_afk is now known as fhubik | 10:44 | |
| *** vinay_yadhav has joined #openstack-infra | 10:44 | |
| *** fhubik is now known as fhubik_afk | 10:46 | |
| openstackgerrit | Sebastien Badia proposed openstack-infra/irc-meetings: puppet: Fix meeting_id of OpenStack Puppet meetings https://review.openstack.org/207389 | 10:48 |
| greghaynes | fungi: I decided to have a look at that openstackid repo when I heard were supposed to be migrating gerrit over to it and I noticed - its lacking a license AFAICT? | 10:48 |
| greghaynes | fungi: pinging you since you have commits there | 10:48 |
| *** teran_ has quit IRC | 10:50 | |
| *** MarkAtwood has joined #openstack-infra | 10:51 | |
| *** jschwarz has joined #openstack-infra | 10:52 | |
| *** _nadya_ has quit IRC | 10:55 | |
| *** _nadya_ has joined #openstack-infra | 10:56 | |
| *** MarkAtwood has quit IRC | 10:56 | |
| *** _nadya_ has quit IRC | 11:00 | |
| odyssey4me | jhesketh ouch, another bad day in openstack-ci :/ | 11:02 |
| *** dims_ has joined #openstack-infra | 11:02 | |
| *** dims has quit IRC | 11:02 | |
| *** davideagnello has joined #openstack-infra | 11:03 | |
| *** jlvillal has joined #openstack-infra | 11:04 | |
| *** mwagner_lap has quit IRC | 11:05 | |
| *** aukhan has quit IRC | 11:06 | |
| *** davideagnello has quit IRC | 11:07 | |
| *** Somay has quit IRC | 11:09 | |
| *** achanda has joined #openstack-infra | 11:10 | |
| *** pal has joined #openstack-infra | 11:11 | |
| *** yfried__ is now known as yfried|afk | 11:12 | |
| *** yfried|afk is now known as yfried__ | 11:12 | |
| *** achanda has quit IRC | 11:15 | |
| *** geoffarnold has quit IRC | 11:17 | |
| pelix | zaro: regarding https://review.openstack.org/#/c/154316, let me know what you think is the best direction | 11:17 |
| *** geoffarnold has joined #openstack-infra | 11:17 | |
| pelix | zaro: similarly https://review.openstack.org/#/c/133774/, think I've answered you previous queries, sorry if that wasn't obvious before. | 11:18 |
| *** tonyb has quit IRC | 11:19 | |
| *** jasondotstar has quit IRC | 11:19 | |
| *** tonyb has joined #openstack-infra | 11:20 | |
| *** deepakcs has quit IRC | 11:25 | |
| *** salv-orl_ has joined #openstack-infra | 11:25 | |
| *** jcoufal has quit IRC | 11:26 | |
| *** salv-orlando has quit IRC | 11:26 | |
| *** jcoufal has joined #openstack-infra | 11:26 | |
| *** pc_m has joined #openstack-infra | 11:36 | |
| *** ldnunes has joined #openstack-infra | 11:36 | |
| *** weshay has joined #openstack-infra | 11:37 | |
| *** koolhead17 has quit IRC | 11:37 | |
| *** jamespage_ has joined #openstack-infra | 11:43 | |
| *** jamespage_ has quit IRC | 11:44 | |
| *** thorst1 has joined #openstack-infra | 11:44 | |
| *** swat30 has quit IRC | 11:45 | |
| *** devkulkarni has joined #openstack-infra | 11:46 | |
| *** swat30 has joined #openstack-infra | 11:47 | |
| *** marcusvrn has joined #openstack-infra | 11:47 | |
| *** e0ne has quit IRC | 11:47 | |
| *** pal has quit IRC | 11:48 | |
| *** banix has joined #openstack-infra | 11:49 | |
| *** hashar has joined #openstack-infra | 11:50 | |
| *** otter768 has joined #openstack-infra | 11:53 | |
| *** gordc has joined #openstack-infra | 11:54 | |
| *** yamamoto has quit IRC | 11:55 | |
| *** bswartz has quit IRC | 11:57 | |
| *** erikwilson_ has joined #openstack-infra | 11:57 | |
| *** otter768 has quit IRC | 11:58 | |
| *** ParsectiX has quit IRC | 12:01 | |
| *** erikwilson_ has quit IRC | 12:01 | |
| *** fawadk has joined #openstack-infra | 12:02 | |
| *** fawadkhaliq has quit IRC | 12:03 | |
| *** dzamboni has joined #openstack-infra | 12:05 | |
| *** _kiran_ has joined #openstack-infra | 12:06 | |
| *** yamamoto has joined #openstack-infra | 12:06 | |
| openstackgerrit | Bharat Kumar Kobagana (BharatK) proposed openstack-infra/project-config: Add experimental glusterfs-manila job https://review.openstack.org/207425 | 12:07 |
| dzamboni | anteaya: ping | 12:09 |
| *** vinay_yadhav has quit IRC | 12:09 | |
| *** kiran-r has quit IRC | 12:09 | |
| *** tlian has joined #openstack-infra | 12:10 | |
| *** fawadk has quit IRC | 12:10 | |
| *** amitgandhinz has joined #openstack-infra | 12:11 | |
| *** _nadya_ has joined #openstack-infra | 12:11 | |
| *** yamamoto has quit IRC | 12:11 | |
| *** dkranz has quit IRC | 12:13 | |
| *** bknudson has joined #openstack-infra | 12:18 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: telemetry: add integration tests job https://review.openstack.org/207298 | 12:18 |
| *** _nadya_ has quit IRC | 12:18 | |
| *** jaosorior has joined #openstack-infra | 12:19 | |
| *** _nadya_ has joined #openstack-infra | 12:19 | |
| *** devkulkarni has quit IRC | 12:19 | |
| *** fhubik_afk is now known as fhubik | 12:20 | |
| *** _nadya_ has quit IRC | 12:20 | |
| *** _kiran_ has quit IRC | 12:20 | |
| *** eharney has quit IRC | 12:21 | |
| *** cburroughs has joined #openstack-infra | 12:22 | |
| *** jasondotstar has joined #openstack-infra | 12:22 | |
| *** amitgandhinz has quit IRC | 12:22 | |
| *** amitgandhinz has joined #openstack-infra | 12:22 | |
| *** ParsectiX has joined #openstack-infra | 12:22 | |
| openstackgerrit | Bharat Kumar Kobagana (BharatK) proposed openstack-infra/project-config: Add experimental glusterfs-manila job https://review.openstack.org/207425 | 12:24 |
| *** ericksonsantos has joined #openstack-infra | 12:25 | |
| *** teran has joined #openstack-infra | 12:25 | |
| openstackgerrit | Anyi Li proposed openstack-infra/jenkins-job-builder: Add value description feature to extended choice parameter https://review.openstack.org/207028 | 12:25 |
| *** samueldmq has joined #openstack-infra | 12:26 | |
| openstackgerrit | Sebastien Badia proposed openstack-infra/irc-meetings: puppet: Fix meeting_id of OpenStack Puppet meetings https://review.openstack.org/207389 | 12:29 |
| *** yamamoto has joined #openstack-infra | 12:30 | |
| *** amitgandhinz has quit IRC | 12:30 | |
| *** zeih has joined #openstack-infra | 12:32 | |
| *** aysyd has joined #openstack-infra | 12:33 | |
| *** cdoron has joined #openstack-infra | 12:34 | |
| *** jamielennox|away is now known as jamielennox | 12:35 | |
| *** fhubik has quit IRC | 12:38 | |
| *** fhubik has joined #openstack-infra | 12:38 | |
| *** bswartz has joined #openstack-infra | 12:39 | |
| *** banix has quit IRC | 12:40 | |
| *** MarkAtwood has joined #openstack-infra | 12:40 | |
| *** cdoron has quit IRC | 12:41 | |
| *** baoli has joined #openstack-infra | 12:41 | |
| *** warpc__ has joined #openstack-infra | 12:41 | |
| *** zeih has quit IRC | 12:42 | |
| *** fhubik has quit IRC | 12:43 | |
| *** baoli has quit IRC | 12:43 | |
| *** baoli has joined #openstack-infra | 12:44 | |
| *** fhubik has joined #openstack-infra | 12:44 | |
| *** fhubik_afk has joined #openstack-infra | 12:44 | |
| *** fhubik_afk is now known as fhubik_real | 12:44 | |
| *** kgiusti has joined #openstack-infra | 12:45 | |
| *** MarkAtwood has quit IRC | 12:45 | |
| *** fhubik_afk has joined #openstack-infra | 12:49 | |
| *** jsavak has joined #openstack-infra | 12:49 | |
| *** fhubik_lunch has joined #openstack-infra | 12:50 | |
| *** fhubik_real has quit IRC | 12:50 | |
| *** fhubik has quit IRC | 12:50 | |
| *** zeih has joined #openstack-infra | 12:51 | |
| *** hashar_ has joined #openstack-infra | 12:51 | |
| *** zz_dimtruck is now known as dimtruck | 12:53 | |
| *** hashar has quit IRC | 12:53 | |
| *** rlandy has joined #openstack-infra | 12:54 | |
| *** e0ne has joined #openstack-infra | 12:54 | |
| *** salv-orl_ has quit IRC | 12:54 | |
| *** hashar_ is now known as hashar | 12:54 | |
| *** fhubik_lunch has quit IRC | 12:54 | |
| *** fhubik_afk has quit IRC | 12:55 | |
| *** fhubik has joined #openstack-infra | 12:55 | |
| *** yamamoto has quit IRC | 12:57 | |
| *** gildub has quit IRC | 12:58 | |
| *** erikwilson_ has joined #openstack-infra | 12:59 | |
| *** kushal has quit IRC | 13:00 | |
| sdague | greghaynes: dib functional tests are pulling content from the internet - http://logs.openstack.org/15/190715/1/check/gate-dib-dsvm-functests-devstack-trusty/2d89fc3/console.html.gz ? | 13:01 |
| sdague | and are co-gating with the rest of openstack | 13:01 |
| sdague | that's kind of bad form in causing gate backups | 13:02 |
| *** davideagnello has joined #openstack-infra | 13:03 | |
| *** katyafervent is now known as katyafervent_awa | 13:04 | |
| *** katyafervent_awa is now known as katyafervent | 13:04 | |
| *** romcheg1 has joined #openstack-infra | 13:04 | |
| *** romcheg has quit IRC | 13:04 | |
| *** akshai has joined #openstack-infra | 13:04 | |
| *** katyafervent is now known as katyafervent_awa | 13:05 | |
| *** katyafervent_awa is now known as katyafervent | 13:05 | |
| openstackgerrit | Sean Dague proposed openstack/diskimage-builder: Add centos7 test https://review.openstack.org/192521 | 13:05 |
| *** markus_z has joined #openstack-infra | 13:05 | |
| *** jamielennox is now known as jamielennox|away | 13:06 | |
| *** eharney has joined #openstack-infra | 13:06 | |
| *** davideagnello has quit IRC | 13:08 | |
| dzamboni | krtaylor: ping | 13:09 |
| *** julim has joined #openstack-infra | 13:10 | |
| *** ig0r_ has quit IRC | 13:10 | |
| *** fawadkhaliq has joined #openstack-infra | 13:11 | |
| *** markus_z has quit IRC | 13:12 | |
| *** btully has joined #openstack-infra | 13:13 | |
| greghaynes | sdague: yes, they have to pull from the internet but co gating with all of openstack seems unnecessary | 13:14 |
| greghaynes | we dont have package mirrors nor image hosting | 13:15 |
| *** zeih has quit IRC | 13:15 | |
| sdague | greghaynes: then the ironic job needs to be deleted off of it | 13:15 |
| sdague | because pulling from the internet is hugely failure prone | 13:15 |
| *** fawadkhaliq has quit IRC | 13:16 | |
| *** ParsectiX has quit IRC | 13:16 | |
| *** ig0r_ has joined #openstack-infra | 13:16 | |
| sdague | greghaynes: is there a patch up for that? | 13:17 |
| *** kushal has joined #openstack-infra | 13:17 | |
| greghaynes | it is, and it would be awesome for us to not have to do it | 13:17 |
| greghaynes | not sure why you say that cant pass? | 13:17 |
| sdague | greghaynes: url? | 13:17 |
| greghaynes | the dib centos 7 test | 13:17 |
| sdague | greghaynes: it's throwing a variable unset failure | 13:17 |
| sdague | because sometime in the last six weeks you introduced -u ? | 13:18 |
| greghaynes | oh fun (im on phone so debugging abilities are limited) | 13:18 |
| sdague | when test results are 6 weeks old, it's better to recheck before approve | 13:18 |
| greghaynes | no, thats been linted as being there forever, must have been some other change that makes use of the var | 13:18 |
| *** pal has joined #openstack-infra | 13:18 | |
| pabelanger | poor zuul, not a good week for you | 13:19 |
| *** dims_ has quit IRC | 13:20 | |
| sdague | greghaynes: yeh, might be | 13:20 |
| *** dims has joined #openstack-infra | 13:20 | |
| *** jsavak has quit IRC | 13:20 | |
| *** marzif_ has quit IRC | 13:21 | |
| sdague | and we hit the apt mirror hash mismatch issue - https://jenkins02.openstack.org/job/gate-tempest-dsvm-full/22692/console | 13:21 |
| sdague | that's going to fail a bunch of stuff randomly | 13:21 |
| *** jsavak has joined #openstack-infra | 13:22 | |
| *** johnbelamaric has joined #openstack-infra | 13:22 | |
| *** e0ne has quit IRC | 13:22 | |
| odyssey4me | sdague it's my understanding that the hash sum mismatch (which we hit relatively often) is caused by the mirror being mid-update | 13:23 |
| *** rbrooker has joined #openstack-infra | 13:23 | |
| sdague | odyssey4me: yep | 13:23 |
| mika | if older debian releases are involved (and or a proxy) then Acquire::http::Pipeline-Depth 0; might help | 13:24 |
| odyssey4me | does infra have any form of apt pre-staging, caching or mirrors? | 13:24 |
| mika | (haproxy, squid,...) | 13:24 |
| sdague | mika: it's trusty | 13:24 |
| mika | shouldn't happen there, then very probably mid-update mirror | 13:24 |
| *** sergmelikyan has joined #openstack-infra | 13:24 | |
| *** woodster_ has joined #openstack-infra | 13:24 | |
| sdague | yeh, the fact that the mirror update protocol ever produces inconsistent mirrors is kind of nuts though | 13:25 |
| greghaynes | odyssey4me: there is caching in our images if your repo supports bindep style other-requirements | 13:25 |
| sdague | odyssey4me: it does, but it updates live as well | 13:25 |
| mika | sdague: jepp, this hashsum mismatch is horrible in CI/CD environments (we use our own mirror therefore, still fight against it there though, might be related to reprepro) | 13:26 |
| *** alivigni has joined #openstack-infra | 13:26 | |
| odyssey4me | greghaynes so we (os-ansible-deployment) have been talking a little about contributing to other-requirements as we already have many, many of the requirements worked out for projects that we have deployments developed for | 13:26 |
| mika | what we™ also do is re-running apt-get in case of failures at least once more | 13:26 |
| odyssey4me | we've done ubuntu 14.04 and rhel/centos 7 I think, which isn't complete, but it's quite a bit of work done | 13:26 |
| *** rfolco has joined #openstack-infra | 13:27 | |
| greghaynes | odyssey4me: you should be able to just propose the patches to the relavent projects | 13:27 |
| odyssey4me | we'd love to make use of bindep to consume the package lists based on the running base OS, instead of maintaining the patches in our own project | 13:27 |
| odyssey4me | what's the penetration of other-requirements through various projects? I haven't seen to many... | 13:28 |
| greghaynes | sdage: so removing the ironic job from dib would mean dib gate failures dont pop the gate stack behind it? | 13:29 |
| greghaynes | I was under the impression that if youre in the gate and you fail you invalidate behind you reguardless of the tests | 13:29 |
| *** jasondotstar has quit IRC | 13:30 | |
| *** tiswanso has joined #openstack-infra | 13:30 | |
| greghaynes | er, jobs, not tests | 13:30 |
| *** mpaolino has quit IRC | 13:30 | |
| *** ig0r_ has quit IRC | 13:31 | |
| greghaynes | If it would fix anything im fine with removing it - I cant remember a sibgle time that test has found something | 13:31 |
| *** e0ne has joined #openstack-infra | 13:31 | |
| *** katyafervent is now known as katyafervent_awa | 13:33 | |
| *** marzif_ has joined #openstack-infra | 13:33 | |
| jroll | greghaynes: fwiw, ironic is working on deprecating the bash ramdisk that the ironic/dib job tests | 13:34 |
| *** katyafervent_awa is now known as katyafervent | 13:34 | |
| johnbelamaric | anteaya: Hi Anita - good morning! I recently had the networking-infoblox repo created, and need myself added to the -core and -release groups | 13:34 |
| greghaynes | jroll: yea, I've been hearing that for a while ;) | 13:35 |
| jroll | greghaynes: I'm about to land https://review.openstack.org/#/c/205142/ :P | 13:35 |
| jroll | itshappening.gif | 13:35 |
| greghaynes | haha, nice | 13:36 |
| dhellmann | fungi, jeblair : are things in a state where we can ask to have the neutron-lbaas-tarball job that failed yesterday re-run, or do you need more time to finish wrapping up the outage? | 13:36 |
| *** ParsectiX has joined #openstack-infra | 13:36 | |
| fungi | dhellmann: sorry, i've not been around much. babysitting more contractors at the house this morning. i suspect the backlog shouldn't impact rerunning a release pipeline job, so i can trigger that for you shortly | 13:37 |
| dhellmann | fungi: great, thanks, do you need anything details? | 13:38 |
| fungi | dhellmann: i'll work it out from the most recent tag on that repo | 13:38 |
| dhellmann | fungi: ok, thanks | 13:38 |
| *** pal has quit IRC | 13:39 | |
| fungi | i take it we're not currently releasing that to pypi, you just need the tarball generated/uploaded | 13:39 |
| *** mjturek1 has joined #openstack-infra | 13:40 | |
| fungi | dhellmann: 7.0.0.0b2 presumably? so pre-release pipeline i guess | 13:41 |
| dhellmann | fungi: yes, that's right | 13:41 |
| fungi | yeah, looks like it. dated yesterday | 13:41 |
| *** glauco_ has joined #openstack-infra | 13:42 | |
| *** glauco_ has quit IRC | 13:42 | |
| *** glauco has joined #openstack-infra | 13:42 | |
| *** mwagner_lap has joined #openstack-infra | 13:42 | |
| *** annegentle has joined #openstack-infra | 13:43 | |
| *** fhubik is now known as fhubik_afk | 13:46 | |
| *** pal has joined #openstack-infra | 13:49 | |
| *** salv-orlando has joined #openstack-infra | 13:49 | |
| *** lykinsbd has joined #openstack-infra | 13:49 | |
| *** dkranz has joined #openstack-infra | 13:51 | |
| *** yfried__ has quit IRC | 13:53 | |
| *** jaypipes has joined #openstack-infra | 13:53 | |
| *** cbader has joined #openstack-infra | 13:54 | |
| *** sinval has joined #openstack-infra | 13:54 | |
| *** otter768 has joined #openstack-infra | 13:54 | |
| *** krtaylor has quit IRC | 13:55 | |
| *** annegentle has quit IRC | 13:55 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: make package install safer https://review.openstack.org/207462 | 13:56 |
| sdague | fungi / greghaynes ^^^ | 13:56 |
| sdague | greghaynes: the gate is dynamically built by transitively computing all the changes which share tests | 13:57 |
| sdague | so the fact that dib and ironic share a test | 13:57 |
| sdague | means it's cogating with nova | 13:57 |
| sdague | because ironic shares tests with nova | 13:57 |
| sdague | and thus the rest of the "gatE" | 13:57 |
| *** ayoung has joined #openstack-infra | 13:57 | |
| *** yamamoto has joined #openstack-infra | 13:58 | |
| *** otter768 has quit IRC | 13:59 | |
| greghaynes | awesome. | 13:59 |
| sdague | greghaynes: where is the proposed change? | 14:00 |
| sdague | I can't find it anywhere | 14:01 |
| greghaynes | to remove the test? I dont think it exists | 14:01 |
| sdague | ok, can you push that change? | 14:01 |
| sdague | because I brought this up a couple of weeks ago as well, and I thought someone was dealing with it | 14:01 |
| greghaynes | when I am at a computer, probably wont be for a few hours | 14:01 |
| sdague | ok | 14:02 |
| *** tvanderwiel has joined #openstack-infra | 14:02 | |
| *** pal has quit IRC | 14:02 | |
| *** yamamoto has quit IRC | 14:03 | |
| *** pal has joined #openstack-infra | 14:03 | |
| greghaynes | we'll need to make sure ironic is ok removing that before they officially deprecate the bash ramdisk | 14:03 |
| greghaynes | jroll ^ | 14:03 |
| *** ninag has joined #openstack-infra | 14:04 | |
| greghaynes | If they really care we can make a test in dib that gets most of the same coverage, but IMO probably not even worth it | 14:04 |
| jroll | greghaynes: we can push a patch for that today :) | 14:04 |
| greghaynes | sweet | 14:04 |
| *** tvanderwiel has left #openstack-infra | 14:04 | |
| *** dkranz has quit IRC | 14:04 | |
| *** jamespage_ has joined #openstack-infra | 14:05 | |
| *** prad has joined #openstack-infra | 14:05 | |
| *** devkulkarni has joined #openstack-infra | 14:05 | |
| sdague | fungi current gate resets are basically being caused by a thing that - https://review.openstack.org/207462 will solve | 14:05 |
| *** amitgandhinz has joined #openstack-infra | 14:05 | |
| *** jamespage_ has quit IRC | 14:05 | |
| sdague | it's going to be chaos for a bit otherwise | 14:05 |
| *** ninag has left #openstack-infra | 14:06 | |
| *** markvoelker_ has quit IRC | 14:06 | |
| *** dguerri is now known as dguerri` | 14:06 | |
| *** krtaylor has joined #openstack-infra | 14:07 | |
| *** ParsectiX has quit IRC | 14:09 | |
| *** dimtruck is now known as zz_dimtruck | 14:09 | |
| fungi | sdague: yep, just testing it out real quick | 14:10 |
| sdague | fungi: coolio | 14:10 |
| openstackgerrit | Marton Kiss proposed openstack-infra/system-config: Add solr dist-root to askbot-staging https://review.openstack.org/207472 | 14:11 |
| sdague | turns out, most of these nodes seem to have python-yaml on them, so it should eval to a noop most of the time. I wasn't confident enough about that fact to gut the path entirely. | 14:11 |
| *** ayoung is now known as ayoung-mtg | 14:11 | |
| marcusvrn | mmedvede: ping | 14:11 |
| *** devkulkarni has quit IRC | 14:11 | |
| fungi | sdague: yeah, looks fine and the return values check out for me | 14:12 |
| sdague | any chance you want to fast promote that? | 14:12 |
| sdague | might get the gate more stable | 14:12 |
| fungi | sdague: yeah, let's get it up to the top | 14:12 |
| sdague | cool | 14:12 |
| mmedvede | marcusvrn: pong | 14:12 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:13 | |
| marcusvrn | mmedvede: hi! You are a third-party coodinator, aren't you? | 14:13 |
| mmedvede | marcusvrn: yes, I am one of a bunch | 14:13 |
| *** achanda has joined #openstack-infra | 14:14 | |
| marcusvrn | mmedvede: could you add a manila ci account to the CI group? | 14:14 |
| *** k4n0 has quit IRC | 14:15 | |
| *** Kennan2 has quit IRC | 14:16 | |
| mmedvede | marcusvrn: sure, what is the account's email? | 14:16 |
| *** hashar_ has joined #openstack-infra | 14:16 | |
| *** Kennan has joined #openstack-infra | 14:16 | |
| *** fhubik_afk is now known as fhubik | 14:17 | |
| marcusvrn | mmedvede: email: OpenstackManilaCI@hds.com ; username: Hitachi-Manila-HNAS-CI | 14:17 |
| *** achanda has quit IRC | 14:18 | |
| *** hashar has quit IRC | 14:19 | |
| *** dkranz has joined #openstack-infra | 14:19 | |
| mmedvede | marcusvrn: done | 14:19 |
| marcusvrn | mmedvede: nice! thanks so much :) | 14:20 |
| mmedvede | marcusvrn: yw =) | 14:20 |
| *** pal has quit IRC | 14:20 | |
| *** ihrachyshka has quit IRC | 14:21 | |
| *** ajmiller has quit IRC | 14:21 | |
| *** sputnik13 has joined #openstack-infra | 14:22 | |
| *** annegentle has joined #openstack-infra | 14:24 | |
| *** zz_dimtruck is now known as dimtruck | 14:27 | |
| *** markvoelker has joined #openstack-infra | 14:27 | |
| *** glauco has quit IRC | 14:27 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-nodepool: Add creation of secure.conf file https://review.openstack.org/188325 | 14:28 |
| *** markvoelker_ has joined #openstack-infra | 14:28 | |
| *** pal has joined #openstack-infra | 14:28 | |
| *** eharney has quit IRC | 14:29 | |
| *** bcourt has quit IRC | 14:29 | |
| *** fhubik has quit IRC | 14:30 | |
| *** ajmiller has joined #openstack-infra | 14:30 | |
| *** tonytan4ever has joined #openstack-infra | 14:30 | |
| *** esmute_ has joined #openstack-infra | 14:31 | |
| *** _nadya_ has joined #openstack-infra | 14:32 | |
| *** ihrachyshka has joined #openstack-infra | 14:32 | |
| *** ig0r_ has joined #openstack-infra | 14:32 | |
| *** markvoelker has quit IRC | 14:32 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for nova unit test race bug 1479780 https://review.openstack.org/207484 | 14:34 |
| openstack | bug 1479780 in OpenStack Compute (nova) "test_describe_instances_with_filters_tags is non deterministic" [High,Confirmed] https://launchpad.net/bugs/1479780 - Assigned to Sean Dague (sdague) | 14:34 |
| *** bcourt has joined #openstack-infra | 14:34 | |
| *** pal has quit IRC | 14:34 | |
| *** BharatK_ has quit IRC | 14:35 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Move hook-{name}-rtfd to proposal node https://review.openstack.org/206618 | 14:37 |
| *** rbrooker has quit IRC | 14:38 | |
| *** arxcruz has quit IRC | 14:38 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-os_client_config: Manage OSCC config file with Puppet https://review.openstack.org/199674 | 14:40 |
| *** masayukig has quit IRC | 14:41 | |
| *** masayukig has joined #openstack-infra | 14:42 | |
| *** salv-orlando has quit IRC | 14:42 | |
| *** bapalm_ has joined #openstack-infra | 14:43 | |
| *** arxcruz has joined #openstack-infra | 14:43 | |
| *** devkulkarni has joined #openstack-infra | 14:44 | |
| *** MarkAtwood has joined #openstack-infra | 14:44 | |
| *** glauco has joined #openstack-infra | 14:44 | |
| *** vlaza has quit IRC | 14:44 | |
| *** eharney has joined #openstack-infra | 14:44 | |
| *** BharatK_ has joined #openstack-infra | 14:46 | |
| *** mtanino has joined #openstack-infra | 14:50 | |
| *** sergmelikyan has quit IRC | 14:50 | |
| *** achanda has joined #openstack-infra | 14:51 | |
| *** sergmelikyan has joined #openstack-infra | 14:51 | |
| NobodyCam | good morning infra folk.. would anyone happen to have a free cycle to review : https://review.openstack.org/#/c/207166 has a +2 from mordred already :) | 14:52 |
| *** soren has quit IRC | 14:53 | |
| *** mase_home has joined #openstack-infra | 14:53 | |
| *** flepied1 has joined #openstack-infra | 14:53 | |
| *** afazekas has quit IRC | 14:55 | |
| *** flepied has quit IRC | 14:56 | |
| *** BobBall is now known as BobBall_AWOL | 14:58 | |
| *** dangers_away has quit IRC | 14:58 | |
| *** masayukig has quit IRC | 14:59 | |
| *** BharatK_ has quit IRC | 14:59 | |
| *** masayukig has joined #openstack-infra | 15:00 | |
| *** austin81 has joined #openstack-infra | 15:00 | |
| *** esmute_ has quit IRC | 15:01 | |
| *** krtaylor has quit IRC | 15:01 | |
| *** jschwarz has quit IRC | 15:01 | |
| *** abregman has joined #openstack-infra | 15:01 | |
| *** sabeen has joined #openstack-infra | 15:01 | |
| fungi | dhellmann: succeeded https://jenkins03.openstack.org/job/neutron-lbaas-tarball/2/consoleFull | 15:02 |
| *** rbrooker has joined #openstack-infra | 15:02 | |
| jeblair | restarted carbon-cache | 15:03 |
| *** ig0r_ has quit IRC | 15:03 | |
| fungi | jeblair: did it oom again? | 15:03 |
| *** jtomasek has quit IRC | 15:03 | |
| *** ig0r__ has joined #openstack-infra | 15:03 | |
| *** achanda has quit IRC | 15:03 | |
| jeblair | fungi: assume so | 15:03 |
| jeblair | seems to be a daily thing now | 15:03 |
| *** banix has joined #openstack-infra | 15:04 | |
| *** BrandonL08 has joined #openstack-infra | 15:04 | |
| NobodyCam | fungi: awesome Thank you :) | 15:04 |
| *** rossella_s has quit IRC | 15:05 | |
| jeblair | fungi: are you/have you taken care of retriggering the pre-release job for dhellmann? | 15:06 |
| fungi | jeblair: yep, see above | 15:06 |
| fungi | jeblair: jhesketh: any idea why manually triggering a job with tools/trigger-job.py and --logpath set is causing "Environment variable 'SWIFT_logs_LOGSERVER_PREFIX' not found"? according to the debug logs SWIFT_logs_LOGSERVER_PREFIX is in the params dict with a value of http://logs.openstack.org/ | 15:07 |
| *** sergmelikyan has quit IRC | 15:07 | |
| *** dkranz has quit IRC | 15:08 | |
| fungi | no, nevermind. i found the wrong log entry | 15:08 |
| fungi | now i get it. trigger-job.py doesn't handle the additional zuul machinery to make swift log uploads work | 15:08 |
| *** ig0r_ has joined #openstack-infra | 15:09 | |
| *** ig0r__ has quit IRC | 15:09 | |
| openstackgerrit | Brandon Leonard proposed openstack-infra/jenkins-job-builder: Adds view functionality https://review.openstack.org/206178 | 15:09 |
| fungi | and since it's injecting the work request straight into gearman, it's not likely much of relevance will appear in the debug logs anyway | 15:10 |
| jeblair | ttx, jhesketh: yes, we need the event to happen one more time, and we'll have the log entries for it. so we should make no changes to our operating practices. | 15:11 |
| *** fawadkhaliq has joined #openstack-infra | 15:11 | |
| BrandonL08 | Hey can someone help me and do a code review for my Jenkins-Job-Builder code? https://review.openstack.org/#/c/206178/ | 15:11 |
| ttx | one more time... | 15:12 |
| ttx | https://www.youtube.com/watch?v=FGBhQbmPwH8 | 15:12 |
| *** ig0r__ has joined #openstack-infra | 15:13 | |
| jeblair | yes that is what needs to happen :) | 15:13 |
| *** glauco has quit IRC | 15:13 | |
| fungi | ttx: if you like leiji matsumoto, you should see the animated film set to that album (interstella 5555) | 15:13 |
| *** krtaylor has joined #openstack-infra | 15:13 | |
| *** unicell has quit IRC | 15:14 | |
| ttx | I'm pretty sure I saw that. | 15:14 |
| russellb | does anyone know if the scripts used to generate lists of ATCs check for Co-authored-by lines? | 15:14 |
| *** ig0r__ has quit IRC | 15:15 | |
| russellb | or know where I can look for code to find out? | 15:15 |
| fungi | ttx: it's an interesting nod to galaxy express 999, the maetel saga, captain harlock, et cetera | 15:15 |
| *** ig0r_ has quit IRC | 15:15 | |
| fungi | russellb: they do not. that' | 15:15 |
| mika | jeblair: might i ping you about https://review.openstack.org/#/c/207239/ and ask why this didn't receive any automated tests yet (shouldn't it)? (ftr: i'm the owner/creator of the patch and IIRC you're taking care of jjb) | 15:15 |
| fungi | s why nova added extra-atcs entries for all the ironic driver code | 15:15 |
| russellb | fungi: ack thanks | 15:16 |
| fungi | russellb: http://git.openstack.org/cgit/openstack-infra/system-config/tree/tools/atc/ | 15:16 |
| fungi | russellb: the script there currently just looks at info in gerrit's database, it does not analyze commit message contents | 15:17 |
| *** MarkAtwood has quit IRC | 15:17 | |
| russellb | ah, i see | 15:17 |
| russellb | probably mostly edge cases anyway | 15:17 |
| *** rossella_s has joined #openstack-infra | 15:17 | |
| jeblair | fungi: omg i had never heard of that, and i love 999 | 15:17 |
| russellb | got an extra-atc proposal related, so just checked | 15:17 |
| *** dannywilson has joined #openstack-infra | 15:17 | |
| *** jcoufal has quit IRC | 15:17 | |
| fungi | jeblair: if you also love daft punk, then you totally need to check it out ;) | 15:17 |
| *** rlandy has quit IRC | 15:18 | |
| clarkb | hey I have actually seen that one | 15:18 |
| fungi | and even if you don't, turn off the sound and it's an interesting silent film (there's no dialogue) | 15:18 |
| *** MarkAtwood has joined #openstack-infra | 15:19 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for nova unit test race bug 1479780 https://review.openstack.org/207484 | 15:19 |
| *** dkranz has joined #openstack-infra | 15:19 | |
| openstack | bug 1479780 in OpenStack Compute (nova) "test_describe_instances_with_filters_tags is non deterministic" [High,Confirmed] https://launchpad.net/bugs/1479780 - Assigned to Sean Dague (sdague) | 15:19 |
| pabelanger | EmilienM: okay, you should be able to rebase a top of 201298 now | 15:20 |
| pabelanger | it was successful under centos7: https://jenkins01.openstack.org/job/gate-puppet-openstack-integration-dsvm-centos7/4/consoleText | 15:21 |
| pabelanger | EmilienM: sorry, wrong channel | 15:21 |
| *** dannywil_ has joined #openstack-infra | 15:22 | |
| *** dannywil_ has quit IRC | 15:22 | |
| openstackgerrit | Merged openstack-infra/glean: Create /root/.ssh folder if folder does not exist https://review.openstack.org/207166 | 15:22 |
| EmilienM | pabelanger: which patch? | 15:22 |
| *** dannywil_ has joined #openstack-infra | 15:22 | |
| *** bhunter71 has joined #openstack-infra | 15:23 | |
| *** dannywi__ has joined #openstack-infra | 15:23 | |
| *** dannywi__ has quit IRC | 15:24 | |
| *** dannywilson has quit IRC | 15:24 | |
| *** dguerri` is now known as dguerri | 15:25 | |
| *** geoffarnold has quit IRC | 15:25 | |
| *** dannywi__ has joined #openstack-infra | 15:26 | |
| *** dannywil_ has quit IRC | 15:27 | |
| *** sdake has joined #openstack-infra | 15:27 | |
| *** austin81 has left #openstack-infra | 15:27 | |
| *** flepied1 has quit IRC | 15:28 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: make package install safer https://review.openstack.org/207462 | 15:29 |
| *** rackertom has quit IRC | 15:29 | |
| *** j^2 has quit IRC | 15:29 | |
| *** xgerman has quit IRC | 15:29 | |
| *** dirk has quit IRC | 15:29 | |
| *** Ng has quit IRC | 15:29 | |
| *** austin81 has joined #openstack-infra | 15:30 | |
| clarkb | jeblair is the log and dont reraise exception during reconfig not also going to fix it by not reraising? | 15:30 |
| clarkb | maybe I misunderstood how that change would help | 15:31 |
| *** zul has quit IRC | 15:31 | |
| *** dannywi__ has quit IRC | 15:32 | |
| jeblair | clarkb: the exception will still break out of the re-enqueue routine, leaving us with some number of changes which have been adjusted to be in the new layout while zuul still runs the old layout (replacing the layout is the last step) | 15:32 |
| *** dannywilson has joined #openstack-infra | 15:34 | |
| *** dguerri is now known as dguerri` | 15:34 | |
| *** dannywilson has quit IRC | 15:35 | |
| *** dannywilson has joined #openstack-infra | 15:36 | |
| *** ifarkas has quit IRC | 15:37 | |
| clarkb | I see | 15:39 |
| clarkb | but it shouldnt stop all work? | 15:39 |
| openstackgerrit | devdatta-kulkarni proposed openstack-infra/project-config: Rename stackforge/solum -> openstack/solum https://review.openstack.org/201287 | 15:39 |
| *** MarkAtwood has quit IRC | 15:40 | |
| *** MarkAtwood has joined #openstack-infra | 15:40 | |
| jeblair | clarkb: it will once the partially-reenqueued change reports -- it is unable to remove that from the queue (since it's not actually in it) | 15:41 |
| *** jasondotstar has joined #openstack-infra | 15:42 | |
| *** hemnafk is now known as hemna | 15:45 | |
| *** rbrooker has quit IRC | 15:45 | |
| *** alexpilotti has quit IRC | 15:48 | |
| *** unicell has joined #openstack-infra | 15:50 | |
| *** skraynev has quit IRC | 15:51 | |
| *** skraynev has joined #openstack-infra | 15:51 | |
| *** MarkAtwood has quit IRC | 15:51 | |
| jeblair | 2015-07-30 15:46:23.270 | The repository located at pypi.dfw.openstack.org is not a trusted or secure host and is being ignored. If this repository is available via HTTPS it is recommended to use HTTPS instead, otherwise you may silence this warning and allow it anyways with '--trusted-host pypi.dfw.openstack.org'. | 15:51 |
| jeblair | http://logs.openstack.org/85/195785/2/gate/gate-zuul-python27/6f79e59/console.html | 15:51 |
| jeblair | that just happened to a zuul change ^ | 15:52 |
| *** vkramskikh has left #openstack-infra | 15:52 | |
| jeblair | is that (a) an actual error, and (b) is that about to start happening everywhere due to new images? | 15:52 |
| *** pc_m is now known as pc_m_AWAY | 15:52 | |
| jeblair | fungi, clarkb: ^ | 15:52 |
| *** MarkAtwood has joined #openstack-infra | 15:52 | |
| *** ig0r_ has joined #openstack-infra | 15:52 | |
| clarkb | we should be setting trusted host but would not be surpised if that is no longer the case with new pip.conf | 15:53 |
| *** maishsk has joined #openstack-infra | 15:54 | |
| jeblair | i'm going to assume that's the case and start deleting images right now | 15:54 |
| clarkb | ok | 15:54 |
| jeblair | https://jenkins05.openstack.org/job/gate-heat-pep8/4205/console | 15:55 |
| clarkb | previously we set trusted host for all 4 nodes regardless of the mirror the job ends up using | 15:55 |
| *** otter768 has joined #openstack-infra | 15:55 | |
| *** woodster_ has quit IRC | 15:55 | |
| *** jaosorior has quit IRC | 15:55 | |
| *** tchaypo has quit IRC | 15:55 | |
| *** ramishra has quit IRC | 15:55 | |
| *** DuncanT has quit IRC | 15:55 | |
| *** dzamboni has quit IRC | 15:55 | |
| *** dalgaaf has quit IRC | 15:55 | |
| *** marcusvrn has quit IRC | 15:55 | |
| *** jswarren has joined #openstack-infra | 15:58 | |
| *** shashank_hegde has joined #openstack-infra | 15:58 | |
| *** nfedotov has joined #openstack-infra | 15:59 | |
| *** rbrooker has joined #openstack-infra | 15:59 | |
| *** glauco_ has joined #openstack-infra | 15:59 | |
| *** amickus has joined #openstack-infra | 15:59 | |
| *** dguerri` is now known as dguerri | 15:59 | |
| zaro | morning | 16:00 |
| *** otter768 has quit IRC | 16:00 | |
| jeblair | okay, all recent images have been deleted | 16:00 |
| *** geoffarnold has joined #openstack-infra | 16:01 | |
| *** rcarrillocruz has quit IRC | 16:01 | |
| dhellmann | fungi, jeblair : thanks! | 16:01 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/puppet-pip/tree/templates/pip.conf.erb#n3 that trusted hosts var may need a @ on it | 16:01 |
| clarkb | nibalizer crinkle ^ | 16:01 |
| crinkle | yes | 16:02 |
| jeblair | i wasn't able to snag a host to confirm | 16:02 |
| clarkb | crinkle would the lack of the @ cause no trsuted hosts to be writteb out? | 16:03 |
| *** glauco_ has quit IRC | 16:03 | |
| clarkb | trying to figure that out now | 16:03 |
| crinkle | clarkb: no, it would just cause a deprecation warning | 16:03 |
| *** puranamr has joined #openstack-infra | 16:04 | |
| clarkb | ok more debugging required. thanks | 16:04 |
| *** jamespage_ has joined #openstack-infra | 16:05 | |
| *** aukhan has joined #openstack-infra | 16:05 | |
| *** rbrooker has quit IRC | 16:06 | |
| *** salv-orl_ has joined #openstack-infra | 16:06 | |
| clarkb | dwf https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/template.pp?id=520fb387e8a445f1f6ead83d93632c40c1c36692#n23 | 16:06 |
| *** shashank_hegde has quit IRC | 16:06 | |
| *** jlanoux has quit IRC | 16:08 | |
| openstackgerrit | David Lyle proposed openstack-infra/system-config: Add searchlight channel to statusbot list https://review.openstack.org/207519 | 16:08 |
| *** MarkAtwood has quit IRC | 16:08 | |
| *** rbrooker has joined #openstack-infra | 16:08 | |
| *** dtantsur is now known as dtantsur|afk | 16:09 | |
| *** MarkAtwood has joined #openstack-infra | 16:09 | |
| *** flepied has joined #openstack-infra | 16:09 | |
| pleia2 | good morning | 16:10 |
| *** maishsk has quit IRC | 16:10 | |
| *** pal has joined #openstack-infra | 16:10 | |
| *** zul has joined #openstack-infra | 16:10 | |
| *** ashleighfarnham has joined #openstack-infra | 16:11 | |
| *** glauco has joined #openstack-infra | 16:12 | |
| zaro | would any infra core willing to approve and babysit this one? https://review.openstack.org/#/c/200727/ | 16:12 |
| *** scheuran has quit IRC | 16:12 | |
| *** sergmelikyan has joined #openstack-infra | 16:12 | |
| zaro | alreayd lots of +1 and two +2 | 16:12 |
| *** sshnaidm has quit IRC | 16:13 | |
| *** davideagnello has joined #openstack-infra | 16:14 | |
| *** rbrooker has quit IRC | 16:14 | |
| *** banix has quit IRC | 16:15 | |
| *** j^2 has joined #openstack-infra | 16:15 | |
| zaro | pelix: looking at those changes | 16:15 |
| *** zul has quit IRC | 16:16 | |
| *** _nadya_ has quit IRC | 16:17 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: puppet: Fix meeting_id of OpenStack Puppet meetings https://review.openstack.org/207389 | 16:17 |
| *** banix has joined #openstack-infra | 16:18 | |
| *** banix has quit IRC | 16:19 | |
| *** banix has joined #openstack-infra | 16:19 | |
| sdague | pep8 install now dying? | 16:20 |
| sdague | --- openstack has changed the topic to: guidelines (Meeting topic: api wg) | 16:21 |
| sdague | <elmiko> #link https://review.openstack.org/#/q/status:open+project:openstack/api-wg,n,z | 16:21 |
| sdague | <elmiko> we had 3 guidelines that will be merging today | 16:21 |
| sdague | <elmiko> 1 already did, but the other 2 aren't for some reason | 16:21 |
| sdague | <sigmavirus24> o/ | 16:21 |
| sdague | <elmiko> we did have one guideline that will need further review | 16:21 |
| sdague | <elmiko> Add new guideline for HTTP Headers #link https://review.openstack.org/#/c/186526/ | 16:21 |
| sdague | <sigmavirus24> elmiko: zuul is about 7.5 hours behind | 16:21 |
| sdague | <elmiko> sigmavirus24: ok, i figured it was something like that | 16:21 |
| *** dalgaaf has joined #openstack-infra | 16:21 | |
| sdague | <sigmavirus24> they notified all channels that they wanted them to not approve things for a while | 16:21 |
| sdague | <elmiko> oops =( | 16:21 |
| sdague | <sigmavirus24> or at least all channels with 'openstackstatus' in it | 16:21 |
| sdague | <elmiko> so, the one guideline that needs further review had some nice comments that came up in the cross project meeting | 16:21 |
| sdague | oops, wrong copy buffer | 16:21 |
| *** liam__ has joined #openstack-infra | 16:21 | |
| sdague | damn you x windows | 16:21 |
| sigmavirus24 | sdague: was I wrong? | 16:21 |
| sigmavirus24 | lol | 16:21 |
| sdague | sigmavirus24: nope | 16:21 |
| sdague | http://paste.openstack.org/ | 16:21 |
| elmiko | hehe | 16:21 |
| *** MarkAtwood has quit IRC | 16:21 | |
| * elmiko waves at sdague | 16:21 | |
| sigmavirus24 | I'm so very confused | 16:21 |
| elmiko | me too | 16:21 |
| sigmavirus24 | sdague: how is pep8's install dying? | 16:22 |
| sigmavirus24 | in the gate? | 16:22 |
| pelix | zaro: thanks | 16:22 |
| sdague | 2015-07-30 15:46:58.756 | The repository located at pypi.dfw.openstack.org is not a trusted or secure host and is being ignored. If this repository is available via HTTPS it is recommended to use HTTPS instead, otherwise you may silence this warning and allow it anyways with '--trusted-host pypi.dfw.openstack.org'. | 16:22 |
| sdague | 2015-07-30 15:46:58.756 | Could not find a version that satisfies the requirement hacking<0.11,>=0.10.0 (from -r /home/jenkins/workspace/gate-ironic-pep8/test-requirements.txt (line 1)) (from versions: ) | 16:22 |
| clarkb | dwf != dfw | 16:22 |
| clarkb | jeblair deleted inages to temporarily fix | 16:22 |
| clarkb | I can push patch after breakfast | 16:22 |
| sdague | ah, oops, I missed that part of scrollback | 16:23 |
| *** jasondotstar has quit IRC | 16:23 | |
| *** MarkAtwood has joined #openstack-infra | 16:23 | |
| *** jistr has quit IRC | 16:24 | |
| clarkb | or anyone else that wants to flip the letters around feel free | 16:25 |
| *** marcusvrn has joined #openstack-infra | 16:26 | |
| *** jaosorior has joined #openstack-infra | 16:27 | |
| *** dirk has joined #openstack-infra | 16:27 | |
| *** zul has joined #openstack-infra | 16:28 | |
| *** flepied1 has joined #openstack-infra | 16:28 | |
| *** aukhan has quit IRC | 16:28 | |
| *** e0ne has quit IRC | 16:29 | |
| *** aukhan has joined #openstack-infra | 16:30 | |
| *** pal has quit IRC | 16:30 | |
| *** Ng has joined #openstack-infra | 16:30 | |
| *** claudiub has quit IRC | 16:31 | |
| sdague | so... I don't feel like the number of nodes reported in nodepool as in use maps to the actual number of active test jobs I'm seeing | 16:31 |
| *** flepied has quit IRC | 16:31 | |
| *** devvesa has quit IRC | 16:33 | |
| *** amitgandhinz has quit IRC | 16:33 | |
| *** woodster_ has joined #openstack-infra | 16:33 | |
| *** dims has quit IRC | 16:33 | |
| *** zul has quit IRC | 16:33 | |
| *** romcheg1 has quit IRC | 16:34 | |
| *** MarkAtwood has quit IRC | 16:37 | |
| *** xgerman has joined #openstack-infra | 16:37 | |
| *** hashar_ has quit IRC | 16:37 | |
| *** MarkAtwood has joined #openstack-infra | 16:37 | |
| *** salv-orl_ has quit IRC | 16:39 | |
| greghaynes | sdague: ok, ironic folks want the dib job to stick around for a bit if possible so im going to copy it (or template it) and rename it so its not coupling to the integrated gate queue | 16:40 |
| greghaynes | jroll: ^ | 16:40 |
| *** shashank_hegde has joined #openstack-infra | 16:40 | |
| *** hichihara has joined #openstack-infra | 16:41 | |
| *** tchaypo has joined #openstack-infra | 16:42 | |
| *** zul has joined #openstack-infra | 16:42 | |
| *** ramishra has joined #openstack-infra | 16:43 | |
| *** DuncanT has joined #openstack-infra | 16:43 | |
| *** rackertom has joined #openstack-infra | 16:43 | |
| zaro | pelix: commented https://review.openstack.org/#/c/15431 | 16:44 |
| *** dangers_away has joined #openstack-infra | 16:45 | |
| *** alexpilotti has joined #openstack-infra | 16:46 | |
| *** ricky1 has joined #openstack-infra | 16:46 | |
| *** skylerberg has joined #openstack-infra | 16:46 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: update governance project list parsing https://review.openstack.org/207541 | 16:47 |
| *** shashank_hegde has quit IRC | 16:47 | |
| *** doude has quit IRC | 16:49 | |
| *** doude has joined #openstack-infra | 16:49 | |
| *** doude has quit IRC | 16:49 | |
| *** doude has joined #openstack-infra | 16:49 | |
| *** MarkAtwood has quit IRC | 16:51 | |
| *** MarkAtwood has joined #openstack-infra | 16:52 | |
| *** shashank_hegde has joined #openstack-infra | 16:53 | |
| *** mpavone has quit IRC | 16:53 | |
| *** esmute_ has joined #openstack-infra | 16:54 | |
| *** nfedotov has quit IRC | 16:54 | |
| sdague | so, something weird just happened | 16:56 |
| sdague | https://review.openstack.org/#/c/207403/ got added to the gate queue | 16:56 |
| sdague | even though jenkins was not +1 on it | 16:56 |
| *** ivar-lazzaro has joined #openstack-infra | 16:57 | |
| *** gordc has quit IRC | 16:57 | |
| *** MarkAtwood has quit IRC | 16:57 | |
| *** dims has joined #openstack-infra | 16:57 | |
| *** ivar-lazzaro has quit IRC | 16:57 | |
| *** shashank_hegde has quit IRC | 16:58 | |
| *** ivar-lazzaro has joined #openstack-infra | 16:58 | |
| sdague | which, is going to be ok, because I tested it quite fine locally | 16:58 |
| *** flepied1 has quit IRC | 16:58 | |
| sdague | but that is an indication of something else being wrong here | 16:58 |
| jeblair | clarkb: some more images finished building, deleting them now | 16:59 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Correct pypi.dfw mirror FQDN in trusted hosts list https://review.openstack.org/207546 | 17:01 |
| fungi | clarkb: jeblair: ^ plain brown wrapper | 17:01 |
| fungi | i also double-checked the others to make sure they're correct | 17:01 |
| fungi | so in theory only images in dfw are broken | 17:01 |
| jeblair | fungi: ah cool, i logged into a system with that on it and looked right at it and didn't see it. i thought i had gotten a machine from an older image. but i still have the terminal up and indeed it is dwf. | 17:02 |
| jeblair | sdague: which thing should i help you with first? nodepool weirdness or zuul weirdness? | 17:02 |
| sdague | so, nodepool weirdness probably impacts more people | 17:03 |
| sdague | the second thing was mostly an FYI | 17:03 |
| *** glauco has quit IRC | 17:03 | |
| jeblair | sdague: (also, i just instapproved 207546 and it went straight to gate) | 17:03 |
| fungi | i'm checking the zuul debug log for clues as to what enqueued 207403 | 17:03 |
| fungi | hrm | 17:03 |
| jeblair | fungi: cool, you can xref that with 207403 :) | 17:03 |
| fungi | so it's happening everywhere | 17:03 |
| *** glauco has joined #openstack-infra | 17:03 | |
| *** abitha has joined #openstack-infra | 17:04 | |
| jeblair | (it's darned convenient ;) | 17:05 |
| *** romcheg has joined #openstack-infra | 17:06 | |
| *** pal has joined #openstack-infra | 17:06 | |
| jeblair | sdague: can you elaborate on the nodepool mismatch? | 17:07 |
| sdague | jeblair: there are 400+ nodes in use | 17:07 |
| *** _nadya_ has joined #openstack-infra | 17:07 | |
| *** dzamboni has joined #openstack-infra | 17:07 | |
| sdague | and clicking around in zuul, I'm not seeing nearly that many in progress runs | 17:07 |
| jeblair | yeah, nodepool list|grep used|wc -l | 17:07 |
| jeblair | 391 | 17:07 |
| *** Slashme has joined #openstack-infra | 17:08 | |
| fungi | chances are we've got a misbehaving jenkins master (or several) again | 17:08 |
| fungi | oh, no that's usually when there are lots of extra ready nodes, not extra used nodes | 17:09 |
| jeblair | yeah, i think we only have about 20 of those | 17:10 |
| fungi | it looks from the graph like zuul might be reporting ~400 jobs running | 17:10 |
| jeblair | er, make that 11 discounting tripleo | 17:10 |
| jeblair | i just saw python take 27 seconds to decode the zuul status yaml; my esteem for javascript interpreters has just increased | 17:12 |
| sdague | heh | 17:13 |
| sdague | maybe I'm wrong jquery says there are 397 progress bars on that page | 17:14 |
| *** sharikapongubala has joined #openstack-infra | 17:14 | |
| sdague | I guess they are just deep in the check queue | 17:14 |
| openstackgerrit | Merged openstack-infra/system-config: Correct pypi.dfw mirror FQDN in trusted hosts list https://review.openstack.org/207546 | 17:14 |
| jeblair | sdague: yeah, we probably have a lot of restarted jobs | 17:15 |
| jeblair | let's declare this one solved then | 17:16 |
| jeblair | https://review.openstack.org/102726 | 17:17 |
| jeblair | fungi, sdague: that merged not to long ago, and touches requirements processing | 17:17 |
| fungi | jeblair: seems possible. i'm not finding a smoking gun in the logs | 17:18 |
| fungi | it basically just logs that it sees the approval trigger, then adds it to the gate pipeline | 17:18 |
| fungi | 2015-07-30 17:03:04,934 DEBUG zuul.DependentPipelineManager: Event <TriggerEvent comment-added openstack-infra/system-config master 207546,1 Workflow:1, Code-Review:2> for change <Change 0x7f4855c1e310 207546,1> matched <EventFilter types: comment-added event_approvals: workflow:1> in pipeline <DependentPipelineManager gate> | 17:19 |
| *** pblaho has quit IRC | 17:19 | |
| jeblair | fungi: matching a pipeline req is not logged (only failing to match one is), so we can't infer much (except that it did not fail to match the requirement if the requirement is there) | 17:21 |
| greghaynes | oh, that patch stack finally merged | 17:21 |
| jeblair | greghaynes: first part of it at least | 17:21 |
| jeblair | the connector stuff got hit with a rebase; it's ready to go, but i thought maybe i'd let things settle down a bit first | 17:21 |
| *** e0ne has joined #openstack-infra | 17:22 | |
| greghaynes | ++ to that | 17:22 |
| jeblair | i'm running a local zuul config test on our config to see if it prints that it thinks there is a pipeline req | 17:22 |
| *** e0ne has quit IRC | 17:23 | |
| *** e0ne has joined #openstack-infra | 17:23 | |
| fungi | jeblair: right, that's what i feared as well. can't exactly log the lack of a need to match something | 17:24 |
| jeblair | INFO:zuul.DependentPipelineManager: <ChangeishFilter open: True current-patchset: True required_approvals: [{'verified': [1, 2], 'username': 'jenkins'}, {'workflow': [1]}]> | 17:24 |
| jeblair | that looks right | 17:24 |
| *** bapalm_ has quit IRC | 17:25 | |
| fungi | yep. and yet that's not present in the query result logged in the debug log | 17:25 |
| *** achanda has joined #openstack-infra | 17:25 | |
| jeblair | oh wait, that doesn't look like what should be output by the current version of zuul... | 17:25 |
| jeblair | (it should now say "required_any_approval") | 17:25 |
| jeblair | i probably have an old zuul in my tox env, whoops | 17:25 |
| *** achanda has quit IRC | 17:26 | |
| *** shashank_hegde has joined #openstack-infra | 17:26 | |
| *** dims has quit IRC | 17:26 | |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Move from stackforge/magnetodb to openstack/magnetodb https://review.openstack.org/203524 | 17:26 |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Changes jobs description for MagnetoDB https://review.openstack.org/204478 | 17:26 |
| fungi | jeblair: yeah, if it's tox on project-config, it doesn't update the zuul checkout there | 17:29 |
| *** achanda has joined #openstack-infra | 17:29 | |
| fungi | and git clean doesn't blow it away for you either | 17:29 |
| fungi | "skipping git repository .test/zuul" or something like that | 17:29 |
| jeblair | INFO:zuul.DependentPipelineManager: <ChangeishFilter open: True current-patchset: True required_any_approval: [{'username': 'jenkins', 'verified': [1, 2]}, {'workflow': 1}]> | 17:30 |
| clarkb | clean with -rfd does clean it iirc | 17:30 |
| *** sputnik13 has quit IRC | 17:30 | |
| jeblair | yeah, so... i think equating the old 'approvals' to 'any approval' was a mistake | 17:31 |
| *** esmute_ has quit IRC | 17:31 | |
| *** ddieterly has joined #openstack-infra | 17:31 | |
| jeblair | i think it's more equvalent to 'all approvals' | 17:31 |
| *** abitha has quit IRC | 17:31 | |
| fungi | clarkb: i don't have a git clean -r (though git clean -dfx is what i usually do and that doesn't blow it away) | 17:32 |
| *** abitha has joined #openstack-infra | 17:32 | |
| clarkb | er xfd | 17:32 |
| jeblair | or maybe not... /me learns what any and all are supposed to refer to here | 17:32 |
| *** sinval has quit IRC | 17:32 | |
| *** _nadya_ has quit IRC | 17:32 | |
| *** ybathia has joined #openstack-infra | 17:32 | |
| *** _nadya_ has joined #openstack-infra | 17:33 | |
| *** _nadya_ has quit IRC | 17:33 | |
| *** esmute_ has joined #openstack-infra | 17:33 | |
| fungi | reminds me of the apt use of those for architectures... all means you build independent binaries for every architecture while any means you run one architecture-independent package on every architecture... or is it the other way around? | 17:33 |
| *** _nadya_ has joined #openstack-infra | 17:33 | |
| fungi | might want to consider deprecating those terms :/ | 17:34 |
| *** sergmelikyan has quit IRC | 17:34 | |
| *** sergmelikyan has joined #openstack-infra | 17:34 | |
| jeblair | they were supposed to be the new terms that made this clear :( | 17:34 |
| jeblair | https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/model.py#n1108 | 17:35 |
| *** erlon has joined #openstack-infra | 17:35 | |
| jeblair | we have a set of required approvals (A) and a set of actual approvals on the change (B) | 17:35 |
| fungi | so "any" is a logical or of approval matches, all is a logical and of approval matches | 17:35 |
| jeblair | our old system was "all items in A must be present in B" | 17:36 |
| *** boris-42 has joined #openstack-infra | 17:37 | |
| greghaynes | yea, that ould be all approvals if I read this right | 17:37 |
| fungi | agreed | 17:38 |
| *** devkulkarni has quit IRC | 17:38 | |
| *** _nadya_ has quit IRC | 17:38 | |
| jeblair | in the new system, the "all" option appears to be "every item in B must be present in A" | 17:39 |
| fungi | clarkb: maybe git clean behavior differs by git version? http://paste.openstack.org/show/406480/ definitely shows it not removing untracked subtrees which are git repositories (git 2.4.6) | 17:39 |
| *** pal has quit IRC | 17:39 | |
| fungi | jeblair: that seems backwards | 17:39 |
| EmilienM | fungi, jeblair: I would your feedback on https://review.openstack.org/206693 - ianw did some suggestion but before changing the patch to copy all /var/log from the publisher, I would like to know what you would do | 17:40 |
| jeblair | and the "any" option appears to be "at least one item in B must be present in A" | 17:40 |
| fungi | hrm. both of those seem backward from what i'd expect then | 17:40 |
| *** ihrachyshka has quit IRC | 17:41 | |
| *** gordc has joined #openstack-infra | 17:41 | |
| fungi | i mean, the net result for "any" is still basically the same i think | 17:41 |
| *** esmute_ has quit IRC | 17:41 | |
| *** sharikapongubala has quit IRC | 17:41 | |
| fungi | but in general i'd expect us to check that a requirement is met, not that something present meets a requirement | 17:41 |
| jeblair | i'm not sure about 'all'; but i'm pretty sure 'any' has a fundamental error | 17:42 |
| jeblair | i think there's even a clue in the code; there's an unused variable | 17:42 |
| fungi | oh, ouch | 17:42 |
| *** Ala has quit IRC | 17:42 | |
| *** sharikapongubala has joined #openstack-infra | 17:43 | |
| *** geoffarnold has quit IRC | 17:43 | |
| fungi | so anyway, i guess we should revert for now, restart zuul, and when reproposing add some more precise scenarios to test these cases? | 17:43 |
| *** bharath has joined #openstack-infra | 17:43 | |
| *** Sukhdev_ has joined #openstack-infra | 17:44 | |
| greghaynes | Hrm, to me all reads that for each required approval were checking that there exists an approval which matches | 17:44 |
| jeblair | greghaynes: for each required approval, we check thet all approvals match | 17:45 |
| greghaynes | ah, yes, because were looping over that to | 17:46 |
| jeblair | yep | 17:46 |
| jeblair | i haven't decided if that's correct or not yet -- it's mostly intended to be used with negative matches, so may be fine | 17:47 |
| fungi | hrm, it does short-circuit with return True on the first match | 17:47 |
| fungi | by my reading | 17:47 |
| greghaynes | fungi: thats only for any | 17:47 |
| fungi | yeah, i was reading through any | 17:47 |
| fungi | oh. you were looking at all | 17:47 |
| *** devkulkarni has joined #openstack-infra | 17:48 | |
| fungi | but yeah, all is implemented backwards, i agree | 17:48 |
| fungi | if there's an approval present which isn't a required approval, it bails | 17:48 |
| fungi | still trying to wrap my head around the negative matching in _match_approval_required_approval() though to figure out how it's supposed to be working | 17:50 |
| *** HeOS has quit IRC | 17:50 | |
| fungi | i guess it's just inverting the match result | 17:50 |
| greghaynes | fungi: ah! yea, we bail too early in the new all matching | 17:51 |
| *** SumitNaiksatam has joined #openstack-infra | 17:51 | |
| johnbelamaric | hello all, I recently had the networking-infoblox repo created, and need myself added to the -core and -release groups (see #link https://review.openstack.org/#/c/203805/) - can anyone help me with that? | 17:51 |
| *** tonytan4ever has quit IRC | 17:52 | |
| *** geoffarnold has joined #openstack-infra | 17:52 | |
| harlowja | mordred so may not be a sync needed, i think scott is just gonna turn on/figure out why its not working http://blog.launchpad.net/general/git-code-hosting-beta for the legacy cloud-init stuff | 17:53 |
| harlowja | mordred i'll let u know, but hopefully can no longer need to do that ever again | 17:53 |
| *** esmute_ has joined #openstack-infra | 17:53 | |
| jeblair | greghaynes, fungi: i'm working on fixing any, and my fix fails its unit tests | 17:53 |
| fungi | i have a feeling the unit tests are also written backwards | 17:53 |
| fungi | oh any, not all | 17:54 |
| openstackgerrit | Aaron Rosen proposed openstack-infra/project-config: Make check-tempest-dsvm-networking-ovn voting https://review.openstack.org/197728 | 17:55 |
| fungi | but even the comments describing matchesRequiredAnyApproval() and matchesRequiredAllApprovals() seem backwards to the implementations i would have expected | 17:56 |
| *** otter768 has joined #openstack-infra | 17:56 | |
| *** alexpilotti has quit IRC | 17:56 | |
| EmilienM | crinkle: what if 1/ we patch all our modules acceptance manifests to put the logs in /openstack/logs (for example) and 2/ patch project-config to cp /openstack/logs into the workspace ? | 17:56 |
| greghaynes | hrm? The any bails as soon as any requirement is found so I think that is right | 17:56 |
| greghaynes | I am getting brain pain trying to digest negative matching though | 17:57 |
| *** BrandonL08 has quit IRC | 17:57 | |
| fungi | greghaynes: yeah, as i said the net result for any is probably the same, but it's described backwards. all is also described/implemented backwards but the net result there is _not_ the same | 17:57 |
| *** esmute_ has quit IRC | 17:57 | |
| *** ayoung-mtg is now known as ayoung | 17:57 | |
| jeblair | greghaynes, fungi: no, it's any that is wrong and is responsible for the behavior we are seeing | 17:57 |
| greghaynes | fungi: im having a hard time thinking of a case where the all returns something different | 17:57 |
| crinkle | EmilienM: how is that different from https://review.openstack.org/#/c/199712/5/spec/acceptance/basic_keystone_spec.rb ? | 17:58 |
| EmilienM | crinkle: we would configure ::keystone::log_dir param to be another path | 17:58 |
| jeblair | greghaynes, fungi: because it's the "wrong" any. it's not supposed to be "any requirement you list" is met, it's supposed to be "any approval left on the change must match all of these things" | 17:58 |
| fungi | jeblair: oh, i assumed any was correct but we should have used all instead, and all has the sets it should be comparing for inclusion swapped | 17:58 |
| *** puranamr has quit IRC | 17:59 | |
| crinkle | EmilienM: why? | 17:59 |
| *** coolsvap|away is now known as coolsvap | 17:59 | |
| crinkle | EmilienM: what about ianw's suggestion on https://review.openstack.org/#/c/206693/ ? | 17:59 |
| EmilienM | crinkle: so we centralize logs in the same place (/openstack/logs) | 17:59 |
| jeblair | tbh, i'm starting to think that maybe we should revert this. its too complex for us to even have a conversation about, and it's a feature that i actively do not want to use. | 17:59 |
| EmilienM | crinkle: he suggests to use a scp publisher, he does not say what to copy | 17:59 |
| fungi | jeblair: but yeah, i can see that "at least one stated requirement is met" is not a useful mechanism | 17:59 |
| * greghaynes was thinking similar to fungi, which I think might be more or less equivilent to what jeblair is saying | 17:59 | |
| EmilienM | crinkle: I suggest to not copy /var/logs/* - it's useless and *heavy* | 17:59 |
| crinkle | EmilienM: I know other projects are doing this sanely, let me see if I can find it in project-config | 17:59 |
| EmilienM | crinkle: I looked ansible | 18:00 |
| EmilienM | crinkle: they put logs in /openstack/logs | 18:00 |
| greghaynes | jeblair: would we need to revert parent patches too? | 18:00 |
| fungi | jeblair: greghaynes: and i assumed "all" was intended to be "all stated requirements are met" but i guess that's what any was supposed to be instead? | 18:00 |
| *** moravec has joined #openstack-infra | 18:01 | |
| jeblair | fungi: yeah, the "any" and "all" are supposed to refer to "the approvals on the change" not "the things listed in the config file" | 18:01 |
| *** otter768 has quit IRC | 18:01 | |
| greghaynes | oh, looks like that one was at the top of a stack, woo | 18:01 |
| greghaynes | heh, so yes then they are swapped | 18:01 |
| EmilienM | crinkle: maybe we could even put the logs in the workspace directly | 18:01 |
| fungi | jeblair: got it. so we wanted any="all stated requirements are met" and all="all stated requirements are met and there are no additional approvals beyond those" | 18:02 |
| EmilienM | crinkle: so we would only need to upload logs to logs.o.o | 18:02 |
| *** hichihara has quit IRC | 18:02 | |
| *** rlucio has quit IRC | 18:03 | |
| jeblair | fungi: yeah, mostly i believe so that you could negate the things in all and say something like "there is no approval that is a -2 code review" | 18:03 |
| greghaynes | sounds like a good test | 18:04 |
| *** salv-orlando has joined #openstack-infra | 18:05 | |
| fungi | got it. i have a feeling modelling the code from the perspective of the configuration rather than from the perspective of the query result would have resulted in less confusing terminology (the sysadmin is more likely to reason about things in terms of what they've configured rather than the other way around) | 18:05 |
| crinkle | EmilienM: it just seems like that's basically what we were doing in https://review.openstack.org/#/c/199712/5/spec/acceptance/basic_keystone_spec.rb so why would we need to add anything to project-config at that point | 18:05 |
| *** esmute_ has joined #openstack-infra | 18:05 | |
| *** ZZelle_ has joined #openstack-infra | 18:06 | |
| pleia2 | pabelanger: reviewing grafana things, if you have a moment to look at my comment on https://review.openstack.org/#/c/201759/ | 18:06 |
| pleia2 | pabelanger: also, has anyone signed up to add the details to hiera for https://review.openstack.org/#/c/179208/ ? or is that me? :) | 18:07 |
| EmilienM | crinkle: my last proposal is not requiring anything from project-config | 18:07 |
| EmilienM | which is good | 18:07 |
| EmilienM | crinkle: but | 18:07 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/project-config: Add repo for apps-catalog-ui https://review.openstack.org/207253 | 18:07 |
| EmilienM | crinkle: it will put the logs in a place that require jenkins installed I guess | 18:07 |
| *** gordc has quit IRC | 18:07 | |
| fungi | jeblair: greghaynes: definitely when looking at the example all and any configs as if they were human language, they read in a way that supports my complete misunderstanding of what they're supposed to imply | 18:08 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Revert "Add support for negative requirements" https://review.openstack.org/207566 | 18:08 |
| crinkle | EmilienM: okay sounds okay to me | 18:09 |
| pabelanger | pleia2: I think that is you. I don't have access, or know who does | 18:09 |
| EmilienM | crinkle: well, that means I can't run beaker on my laptop now | 18:09 |
| jeblair | fungi, greghaynes: yeah, even though i can fix the logic in the code, the tests still need to be fixed, and i agree, this is really hard to explain, which is not what we're going for in zuul. | 18:09 |
| pleia2 | pabelanger: it would be a root person, just wanted to be sure no one eles had been tapped to do it already | 18:10 |
| crinkle | EmilienM: what why? | 18:10 |
| pabelanger | pleia2: good call | 18:10 |
| *** armax has quit IRC | 18:10 | |
| pabelanger | fixing | 18:10 |
| crinkle | EmilienM: in https://review.openstack.org/#/c/199712/5/spec/acceptance/basic_keystone_spec.rb we have it looking for an env varialbe before copying logs | 18:10 |
| *** pvaneck has joined #openstack-infra | 18:10 | |
| pabelanger | pleia2: no, I have not asked anybody | 18:10 |
| *** gordc has joined #openstack-infra | 18:10 | |
| jeblair | (the language looks like "pipeline: gate; require: approval: workflow=1;" so that it's human readable) | 18:10 |
| EmilienM | crinkle: ok so you want a conditional to define the log dir? | 18:11 |
| jeblair | jhesketh: FYI https://review.openstack.org/207566 and last half-hour or so of scrollback | 18:11 |
| EmilienM | crinkle: if ENV['WORKSPACE'] then log_dir = /var/lib/jenkins/... | 18:11 |
| EmilienM | crinkle: else undef | 18:11 |
| *** johnbelamaric has quit IRC | 18:11 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Migrate grafana.o.o to puppet-httpd https://review.openstack.org/201759 | 18:11 |
| crinkle | EmilienM: ew no | 18:11 |
| jeblair | fungi: good news is, that should merge soon :) | 18:11 |
| EmilienM | crinkle: sorry I misunderstood | 18:11 |
| EmilienM | crinkle: what do you suggest? | 18:11 |
| fungi | jeblair: indeed. it has been sped along by it the very bug it's reverting | 18:12 |
| jeblair | fungi: i've enjoyed having the direct-to-gate configuration again. it's actually one of the things i'm most looking forward to with zuulv3 multi-tenancy (i intend to drop that requirement for infra) | 18:12 |
| crinkle | EmilienM: let's make the tests copy the logs to the right place, /openstack/logs or whatever, like we had it many patchsets back in https://review.openstack.org/#/c/199712/5/spec/acceptance/basic_keystone_spec.rb, and then we can have the scp macro publish the logs wherever | 18:12 |
| EmilienM | crinkle: https://review.openstack.org/#/c/199712/5/spec/acceptance/basic_keystone_spec.rb, never did that, I was harcoding the copy/paste at the end of the test | 18:13 |
| fungi | jeblair: greghaynes: perhaps s/any-approval/approvals/ and s/exact-approvals/ or something would be a little clearer from the config end | 18:13 |
| fungi | er, s/all-approvals/exact-approvals/ | 18:13 |
| *** dkranz has quit IRC | 18:14 | |
| jeblair | fungi: is sounding better | 18:14 |
| *** MarkAtwood has joined #openstack-infra | 18:14 | |
| fungi | plus fixing the broken code | 18:14 |
| *** teran has quit IRC | 18:15 | |
| EmilienM | crinkle: I'll do that: patch basic_keystone_spec to create /openstack/logs, put our logs in it and patch infra to scp the logs from this dir | 18:16 |
| EmilienM | crinkle: if you like | 18:17 |
| crinkle | EmilienM: i'm reading ianw's comment more thoroughly | 18:18 |
| crinkle | EmilienM: it seems like he suggests putting all the logic to copy things in project-config, like http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n944 and http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n771 | 18:18 |
| crinkle | EmilienM: if we had a macro that maybe took arguments for which logs to copy that would be better than doing it in the tests | 18:19 |
| EmilienM | crinkle: yeah and I thought about that today - I see two options: 1/ copy /var/log/* -> I don't like that, it's too much or 2/ copy /var/log/<project_name>/* -> better but still it won't copy apache logs when running wsgi | 18:20 |
| crinkle | EmilienM: right, but could we make it take arbitrary arguments? | 18:21 |
| EmilienM | crinkle: sent by who? | 18:21 |
| EmilienM | crinkle: our modules? | 18:21 |
| crinkle | EmilienM: i'm not an expert in jjb but I'm banking on some way the job can pass arguments to the macro | 18:22 |
| *** johnbelamaric has joined #openstack-infra | 18:22 | |
| EmilienM | crinkle: it does but how to make it specific for our modules - or maybe we should not and do something standard | 18:23 |
| EmilienM | crinkle: I can do a macro that copy both wsgi logs & <project-name/*.log ? | 18:23 |
| EmilienM | this solution is really simple | 18:23 |
| *** ig0r__ has joined #openstack-infra | 18:24 | |
| nibalizer | EmilienM: hi | 18:26 |
| EmilienM | nibalizer: o/ | 18:26 |
| nibalizer | I'm reading scrollback, what are you trying to do? | 18:26 |
| *** patrickeast has joined #openstack-infra | 18:26 | |
| *** dims has joined #openstack-infra | 18:26 | |
| EmilienM | nibalizer: copy some useful logs from our beaker runs to job workspace and be able to debug when a job fails | 18:26 |
| *** jtomasek has joined #openstack-infra | 18:26 | |
| *** nm_ has joined #openstack-infra | 18:27 | |
| nibalizer | so a publisher | 18:27 |
| EmilienM | true | 18:27 |
| nibalizer | oh but the beaker indirector makes that sortof complicated | 18:27 |
| EmilienM | nibalizer: but the question is what and how to publush | 18:27 |
| EmilienM | publish* | 18:27 |
| *** dkranz has joined #openstack-infra | 18:27 | |
| nibalizer | which logs do you want? | 18:27 |
| crinkle | nibalizer: that's the issue, the logs will be differetn for each module | 18:28 |
| clarkb | you can use glbs | 18:28 |
| crinkle | for keystone for example it will be /var/log/keystone/* and /var/log/{apache2|httpd}/* | 18:28 |
| *** glauco has quit IRC | 18:28 | |
| clarkb | so copy all to logs/*.txt | 18:28 |
| EmilienM | clarkb: all? | 18:28 |
| clarkb | and publisg that. this is what devstack gate does | 18:29 |
| EmilienM | clarkb: /var/log/* ? | 18:29 |
| clarkb | EmilienM whatever your logs are | 18:29 |
| clarkb | also possibly use d-g | 18:29 |
| nm_ | hi i am trying rasselin CI, one problem i am facing is that, when many jobs are in queue, then when one job is finished,then next job is not waiting for old vm to delete, | 18:29 |
| EmilienM | clarkb: glbs ? | 18:29 |
| clarkb | globs | 18:30 |
| *** jasondotstar has joined #openstack-infra | 18:30 | |
| clarkb | * | 18:30 |
| *** MarkAtwo_ has joined #openstack-infra | 18:30 | |
| nm_ | next job is running on old vm | 18:30 |
| nibalizer | ya whatever d-g does is probably what we should do here | 18:30 |
| *** glauco has joined #openstack-infra | 18:30 | |
| clarkb | nibalizer you can intheory just use that function from d-g | 18:30 |
| *** sharikap_ has joined #openstack-infra | 18:30 | |
| nibalizer | yea | 18:30 |
| nm_ | how can i stop a job to run on used nodepool vm | 18:30 |
| nibalizer | or just the d-g publisher if it exists | 18:31 |
| EmilienM | logs are not in the same place AFIK | 18:31 |
| EmilienM | in devstack, logs are in /opt/stack/logs iirc | 18:31 |
| *** jraim_ has joined #openstack-infra | 18:31 | |
| *** Somay has joined #openstack-infra | 18:31 | |
| *** briancurtin_ has joined #openstack-infra | 18:31 | |
| *** tiswanso has quit IRC | 18:31 | |
| nm_ | any idea? | 18:31 |
| EmilienM | in puppet, we have /var/log/<...> | 18:31 |
| clarkb | nm_ you can use a zuul function to set the single use slave flag for thw gearman plugin | 18:31 |
| *** sweston_ has joined #openstack-infra | 18:32 | |
| clarkb | er the offline on job start flag | 18:32 |
| *** thingee_ has joined #openstack-infra | 18:32 | |
| nibalizer | what about using the devstack-logs publisher ? | 18:32 |
| *** coolsvap is now known as coolsvap|away | 18:32 | |
| clarkb | EmilienM then change the prefix? there are many options here | 18:32 |
| nibalizer | I guess that uses logs/** | 18:32 |
| nibalizer | which is not what we need | 18:32 |
| *** ayoung has quit IRC | 18:32 | |
| *** pelix1 has joined #openstack-infra | 18:32 | |
| *** pelix has quit IRC | 18:33 | |
| *** tiswanso has joined #openstack-infra | 18:33 | |
| clarkb | my point ia more to lwverage what already works than to reinvent a beaker specific wheel | 18:33 |
| EmilienM | clarkb: I 100% agree | 18:33 |
| *** amickus has quit IRC | 18:33 | |
| nibalizer | right and beaker doesn't need to be considered really | 18:33 |
| EmilienM | if I can reuse what we already have in place, I'm all in | 18:33 |
| *** marcusvrn has quit IRC | 18:33 | |
| EmilienM | this is why I'm talking here | 18:33 |
| nibalizer | you can just have a publisher after the job that goes to /var/log/ and grabs the right stuff | 18:33 |
| clarkb | nibalizer yes that would work too | 18:34 |
| EmilienM | nibalizer: yes. But how do you know the right stuff? | 18:34 |
| nibalizer | one question: clarkb is /var/log/** too much stuff for infra to store? | 18:34 |
| *** gordc has quit IRC | 18:34 | |
| nibalizer | thats liable to have all kinds of junk | 18:34 |
| *** johnbelamaric has quit IRC | 18:34 | |
| *** jlvillal_ has joined #openstack-infra | 18:34 | |
| clarkb | yes dont do that | 18:34 |
| crinkle | nibalizer: so we have a job like http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/puppet-module-jobs.yaml#n27 and a project like http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/projects.yaml#n2370 and that project needs specific logs, how do we tell the publisher which ones to copy | 18:34 |
| *** briancurtin has quit IRC | 18:34 | |
| *** dzamboni has quit IRC | 18:34 | |
| *** MarkAtwood has quit IRC | 18:34 | |
| *** dalgaaf has quit IRC | 18:34 | |
| *** thingee has quit IRC | 18:34 | |
| *** sweston has quit IRC | 18:34 | |
| *** jraim has quit IRC | 18:34 | |
| *** ericksonsantos has quit IRC | 18:34 | |
| *** jlvillal has quit IRC | 18:34 | |
| *** eantyshev has quit IRC | 18:34 | |
| *** dkranz has quit IRC | 18:34 | |
| *** sobersabre has quit IRC | 18:34 | |
| *** dingyichen has quit IRC | 18:34 | |
| *** ildikov has quit IRC | 18:34 | |
| *** mjturek1 has quit IRC | 18:34 | |
| *** sharikapongubala has quit IRC | 18:34 | |
| EmilienM | I suggested to crinkle to grap /var/log/<project_name/ and /var/log/<apache_or_httpd/ at leas | 18:34 |
| *** johnbelamaric has joined #openstack-infra | 18:34 | |
| *** ericksonfgds has joined #openstack-infra | 18:34 | |
| clarkb | be more specifix | 18:34 |
| *** briancurtin_ is now known as briancurtin | 18:34 | |
| *** ildikov has joined #openstack-infra | 18:34 | |
| *** dkranz has joined #openstack-infra | 18:35 | |
| nibalizer | crinkle: my thoughts are to just grab everything every time and not worry | 18:35 |
| crinkle | nibalizer: clarkb says don't do that | 18:35 |
| clarkb | nibalizer no | 18:35 |
| *** thingee_ is now known as thingee | 18:35 | |
| clarkb | that can get huge gigabytes huge | 18:35 |
| clarkb | because mysql and rabbit and apache | 18:35 |
| nibalizer | gross okay | 18:35 |
| *** jraim_ is now known as jraim | 18:35 | |
| *** sweston_ is now known as sweston | 18:35 | |
| *** jlvillal_ is now known as jlvillal | 18:35 | |
| crinkle | clarkb: question above ^ | 18:35 |
| *** mjturek1 has joined #openstack-infra | 18:35 | |
| *** dingyichen has joined #openstack-infra | 18:35 | |
| EmilienM | I suggest copying what we really need for now : openstack logs + apache logs | 18:36 |
| nibalizer | well I reccomend you write/use a publisher that has a list of files to pull, start with /var/log/apache/*error.log and keep adding files to the list over time | 18:36 |
| nibalizer | doesn't seem very hard | 18:36 |
| clarkb | I would honestly set thibgs up to lg in a prefix and copy all of that | 18:36 |
| clarkb | vasically what devstack does | 18:36 |
| *** pelix1 is now known as pelix | 18:36 | |
| crinkle | clarkb: so you would have the tests copy the files to a known place? | 18:37 |
| EmilienM | clarkb: what I initially suggested | 18:37 |
| clarkb | so /var/log/openstackor similar | 18:37 |
| clarkb | no | 18:37 |
| EmilienM | to copy logs in /openstack/logs | 18:37 |
| clarkb | no | 18:37 |
| clarkb | either a) config to writr tgere in the first place orb) have jenkins but not the tests xopy them | 18:37 |
| clarkb | thats a subtle difference but it is important | 18:37 |
| crinkle | clarkb: we're asking how to have jenkins copy them | 18:38 |
| clarkb | and my response earlier was use devstack gate | 18:38 |
| clarkb | as an alternative just add a shell builder to do it but that will likely look like d-g in the end | 18:38 |
| EmilienM | clarkb: may I ask you to give an example in d-g ? I just want to make sure I got it | 18:40 |
| *** salv-orlando has quit IRC | 18:40 | |
| nibalizer | ya see the cleanup_host function in d-g | 18:40 |
| *** ricky1 is now known as rcarrillocruz | 18:40 | |
| clarkb | http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n600 | 18:41 |
| jesusaurus | crinkle: you can create a publisher like http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n783 but with a different source given to copy a different set of files | 18:41 |
| nibalizer | I think you still need some shell builder to move selected files out of /var/log/ into a central logs dir before you run that publisher though | 18:42 |
| crinkle | jesusaurus: that sounds like we'd need to create an individual publisher for each module then | 18:42 |
| clarkb | nibalizer if you want large globs yes | 18:42 |
| *** shashank_hegde has quit IRC | 18:42 | |
| EmilienM | clarkb: this code is really what we need, it's cool | 18:42 |
| jesusaurus | nibalizer: oh, true, that source is rooted in $WORKSPACE | 18:42 |
| EmilienM | it's just copying openstack log in the devstack way | 18:42 |
| *** dzamboni has joined #openstack-infra | 18:42 | |
| *** dalgaaf has joined #openstack-infra | 18:42 | |
| EmilienM | ie: not using /var/log/keystone (for example) | 18:43 |
| nibalizer | crinkle: just one shell script kinda like that function in d-g to push things into $WORKSPACE/logs and then that publisher | 18:43 |
| nibalizer | that shell script will probably try to copy files that aren't there, but thats no big deal | 18:43 |
| nibalizer | just don't set -x it | 18:43 |
| nibalizer | er set -e i mean | 18:43 |
| *** changbl has quit IRC | 18:43 | |
| clarkb | as an aside openstack should really log to /var/log/openstack and put config in /etc/openstack to make it easuer to find thing and copy them around | 18:44 |
| clarkb | but last I saw it isnt possible to change those paths for some projects so I left it there | 18:44 |
| *** tonytan4ever has joined #openstack-infra | 18:44 | |
| EmilienM | ok so if I summarize, I need to write a shell script, inspired by http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n600 and adapt it to our Puppet log path | 18:46 |
| nm_ | clarkb : do you have any reference or link to "use a zuul function to set the single use slave flag for thw gearman plugin"? | 18:46 |
| crinkle | EmilienM: i think clarkb suggests to use that function itself | 18:47 |
| *** johnbelamaric has quit IRC | 18:47 | |
| crinkle | if i understand correctly | 18:47 |
| clarkb | crinkle ya, we can change the path it uses from BASE/logs to default to that but be overrideable | 18:47 |
| clarkb | then you can say /var/log is base or whatever | 18:47 |
| *** sharikap_ has quit IRC | 18:48 | |
| EmilienM | sounds cool | 18:48 |
| EmilienM | I'll give a try - thanks clarkb, nibalizer and crinkle | 18:48 |
| clarkb | nm_ https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n34 is wjere we do it | 18:48 |
| clarkb | but this is a gwneral problem testing and debugging openstack | 18:49 |
| clarkb | we have let every projwct do its own thing deciding whwre to store this stuff | 18:49 |
| clarkb | which makes it hard to track it all down and is why the devstack method is sane as it forces it into the foreground where it is controlled | 18:49 |
| *** ig0r_ has quit IRC | 18:50 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Revert "Add support for negative requirements" https://review.openstack.org/207566 | 18:51 |
| jeblair | fungi: you'd think gerrit could generate a revert commit that it could actually merge.... but you'd be wrong. | 18:51 |
| *** hdd has joined #openstack-infra | 18:51 | |
| fungi | jeblair: oh, lemme guess it generated it with the commit being reverted as the parent, rather than the branch tip | 18:51 |
| *** flepied has joined #openstack-infra | 18:52 | |
| jeblair | yep | 18:52 |
| fungi | downright silly | 18:52 |
| jesusaurus | clarkb: ive noticed that my log-gearman-worker.py processes have been slowly increasing their ram footprint, how much memory do you typically see those processes use? | 18:52 |
| *** amitgandhinz has joined #openstack-infra | 18:52 | |
| jeblair | jesusaurus: a lot http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1270&rra_id=all | 18:54 |
| jesusaurus | jeblair: oh, so a gig per process is actually pretty typical, thanks | 18:55 |
| jeblair | jesusaurus: looks like 36% logstash; 16% ES; 15+8+5+4% log-gearman-worker | 18:55 |
| jeblair | (they seem to be unevenly picking up jobs) | 18:55 |
| jeblair | jesusaurus: the rss for the largest is 624m (virt is 900m) | 18:55 |
| nm_ | clarkb: thanks, i will look into it, in my case, when a job is complete, nodepool vm is getting delete, but its taking some time but jobs in queue is not waiting for used vm to delete, so what you told will work for this? | 18:56 |
| *** changbl has joined #openstack-infra | 18:56 | |
| *** Hal has joined #openstack-infra | 18:56 | |
| *** Hal is now known as Guest49146 | 18:57 | |
| nm_ | clarkb: i want jobs in queue to not take used vm | 18:57 |
| clarkb | nm_ what that function does is force the slave offline while the job completes. the when the job is done no new joba can eun on it because it is offline | 18:57 |
| *** MarkAtwo_ is now known as MarkAtwood | 18:59 | |
| *** puranamr has joined #openstack-infra | 18:59 | |
| *** bapalm_ has joined #openstack-infra | 19:00 | |
| *** teran has joined #openstack-infra | 19:01 | |
| *** johnbelamaric has joined #openstack-infra | 19:02 | |
| *** teran has quit IRC | 19:03 | |
| nm_ | clarkb: thanks, how could i use this zuul function, means we have nodepool.yaml, layout.yaml, jobs.yaml do i have to export OFFLINE_NODE_WHEN_COMPLETE somewhere | 19:03 |
| *** ig0r_ has joined #openstack-infra | 19:04 | |
| *** puranamr has quit IRC | 19:04 | |
| *** dkranz has quit IRC | 19:05 | |
| fungi | nm_: you need to set that file as included in your layout.yaml, and set it in a function which you then reference from a parameter-function key in the jobs list | 19:05 |
| clarkb | in your layout.yaml you include the functions file you can see this at the top of our layou file https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml | 19:05 |
| *** ivar-laz_ has joined #openstack-infra | 19:05 | |
| *** bapalm_ has quit IRC | 19:05 | |
| clarkb | then you apply the functio itself to you jobs in the job listing | 19:06 |
| *** ivar-laz_ has quit IRC | 19:06 | |
| *** ddieterly has quit IRC | 19:06 | |
| *** ivar-laz_ has joined #openstack-infra | 19:06 | |
| *** liam__ has quit IRC | 19:07 | |
| *** ivar-lazzaro has quit IRC | 19:08 | |
| EmilienM | clarkb: so I'm reading functions.sh and I don't see I can override the openstack log path to be /var/log/keystone (for example). Do you think i'll need to change the log dir in our puppet CI jobs? | 19:10 |
| *** geoffarnold has quit IRC | 19:11 | |
| clarkb | it isnt possible today. my suggestion was to update that function to make it possible | 19:11 |
| clarkb | assuming that meets your needs | 19:11 |
| *** uberjay has quit IRC | 19:11 | |
| *** uberjay has joined #openstack-infra | 19:12 | |
| EmilienM | ok cool | 19:12 |
| openstackgerrit | Christopher Aedo proposed openstack-infra/project-config: Add repo for apps-catalog-ui https://review.openstack.org/207253 | 19:13 |
| *** marcusvrn has joined #openstack-infra | 19:14 | |
| *** rossella_s has quit IRC | 19:16 | |
| *** rossella_s has joined #openstack-infra | 19:16 | |
| *** mwagner_lap has quit IRC | 19:16 | |
| openstackgerrit | Merged openstack-infra/zuul: Revert "Add support for negative requirements" https://review.openstack.org/207566 | 19:16 |
| *** BrandonL08 has joined #openstack-infra | 19:17 | |
| nm_ | fungi clarkb : thanks, this is what i have http://paste.openstack.org/show/406488/ is this fine for that | 19:17 |
| *** dkranz has joined #openstack-infra | 19:18 | |
| *** warpc__ has quit IRC | 19:18 | |
| *** hdd has quit IRC | 19:19 | |
| fungi | nm_: yeah, as long as your job names match the regex pattern you set there, and as long as you also add the include statement in your layout.yaml like we have at the top of ours | 19:20 |
| fungi | jeblair: zuul.o.o has the revert installed now. should we go ahead and restart or wait? | 19:21 |
| *** HeOS has joined #openstack-infra | 19:21 | |
| jeblair | fungi: my feeling is that it's not worth the disruption to the backlog to restart; but i could be convinced otherwise. | 19:23 |
| *** sabeen has quit IRC | 19:23 | |
| fungi | jeblair: weighing that and the probability that our bug friend might resurface again soon and cause us to need to restart anyway, i think it's safe to let it churn through what it can in the meantime | 19:23 |
| jeblair | ++ | 19:24 |
| *** gordc has joined #openstack-infra | 19:25 | |
| *** e0ne has quit IRC | 19:25 | |
| *** julim has quit IRC | 19:27 | |
| *** sabeen1 has joined #openstack-infra | 19:27 | |
| *** aukhan has quit IRC | 19:28 | |
| *** jasondotstar has quit IRC | 19:29 | |
| *** salv-orlando has joined #openstack-infra | 19:29 | |
| *** kushal has quit IRC | 19:31 | |
| *** armax has joined #openstack-infra | 19:31 | |
| *** sabeen1 has quit IRC | 19:32 | |
| *** kushal has joined #openstack-infra | 19:32 | |
| jroll | greghaynes: rename sounds good to me, thanks | 19:35 |
| EmilienM | crinkle, clarkb: I was hacking in functions.sh and it's very ugly. conditionals, etc... I honestly think devstack-gate script is locked to devstack and it would require many changes to work in puppet jobs | 19:35 |
| *** armax has quit IRC | 19:36 | |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Use automaton 0.5.0 https://review.openstack.org/207589 | 19:37 |
| pleia2 | just to be clear, merging this change to add grafana.o.o today doesn't do much until I actually get the server up (tomorrow), right? https://review.openstack.org/#/c/179208/ ...it will publish docs, and add module, but it shouldn't be an issue otherwise | 19:38 |
| pleia2 | I haven't set up the trove instance yet, so I do need to do that and add other private data, but figured I'd do that tomorrow too | 19:38 |
| openstackgerrit | Khai Do proposed stackforge/python-jenkins: Rework the examples https://review.openstack.org/207590 | 19:38 |
| jeblair | pleia2: yep, it's safe | 19:38 |
| pleia2 | cool, I'll get this in now then so we're ready | 19:39 |
| *** Somay has quit IRC | 19:39 | |
| clarkb | EmilienM just do log_prefix={$1:-$BASE/logs} | 19:39 |
| clarkb | then you can changw it | 19:39 |
| jeblair | pleia2: and i'd recommend doing that, that way once you update hiera with the db info, you can just launch it and it will magically dt[rw]t | 19:39 |
| pleia2 | jeblair: right, that was my thinking, thanks! | 19:39 |
| EmilienM | clarkb: this is (kind of) what I did but still there is $PROJECTS to buil up | 19:39 |
| EmilienM | build* | 19:39 |
| *** tiswanso has quit IRC | 19:40 | |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Use automaton 0.5.0 https://review.openstack.org/207589 | 19:40 |
| fungi | i'm running through the jenkins masters now looking for any which are sickly. i see signs that at least some are, which could be impacting our worker turnover/overhead | 19:41 |
| openstackgerrit | Merged openstack-infra/project-config: added project stackforge/monasca-log-api https://review.openstack.org/194595 | 19:41 |
| jeblair | fungi, clarkb: i'd like you to weigh in on https://review.openstack.org/198236 when you have a min | 19:41 |
| jeblair | pleia2: ^ you too | 19:41 |
| *** AJaeger has joined #openstack-infra | 19:41 | |
| *** fawadkhaliq has quit IRC | 19:42 | |
| AJaeger | EmilienM: have you seen http://lists.openstack.org/pipermail/openstack-infra/2015-July/002966.html ? Do you want to limit creation of branches to a smaller team to reduce the risk? | 19:42 |
| EmilienM | AJaeger: yes, to crinkle and I | 19:43 |
| EmilienM | AJaeger: I'm going to submit a patch | 19:43 |
| AJaeger | EmilienM: Great, thanks! | 19:43 |
| EmilienM | crinkle: you ok? | 19:43 |
| *** fawadkhaliq has joined #openstack-infra | 19:43 | |
| crinkle | EmilienM: sure | 19:43 |
| EmilienM | AJaeger: thanks for pointing that | 19:43 |
| *** marzif_ has quit IRC | 19:43 | |
| pabelanger | annegentle: do you still have commit access to https://github.com/trystack/trystack_org_webcontents ? | 19:44 |
| *** Sukhdev_ has quit IRC | 19:44 | |
| *** marzif_ has joined #openstack-infra | 19:44 | |
| *** kushal has quit IRC | 19:45 | |
| EmilienM | clarkb: the challenge is to build $PROJECTS without hardcoding everything but I'll find something to get from devstack maybe a default value with all projects | 19:45 |
| *** ayoung has joined #openstack-infra | 19:46 | |
| fungi | i've got jenkins01 and jenkins05 in prepare for shutdown now. jenkins06 needs help as well, but i'm uncomfortable taking three offline at the same time so i'll do that once the other two are cleaned up and restarted | 19:46 |
| clarkb | d-g does set PROJECTS though | 19:46 |
| clarkb | so maybe we make that better | 19:46 |
| *** openstackgerrit has quit IRC | 19:46 | |
| *** openstackgerrit has joined #openstack-infra | 19:47 | |
| AJaeger | jeblair, fungi, clarkb, pleia2, anteaya, ianw, sdague, jhesketh : I'll be out until Tuesday and might be busy the rest of the week. so, I cannot ping jhesketh if Zuul is stuck during my morning ;) Hope it's not needed.. | 19:47 |
| jeblair | i hope it _is_ needed | 19:47 |
| jeblair | we need it to get stuck once more so that we can fix it | 19:48 |
| fungi | (so that we can finally find out what's breaking it) | 19:48 |
| mtreinish | anteaya, AJaeger: where/how do I have to add the definition for the branches on: https://review.openstack.org/207055 | 19:49 |
| *** skylerberg has quit IRC | 19:49 | |
| *** achanda has quit IRC | 19:49 | |
| openstackgerrit | Merged openstack-infra/project-config: Update jenkins jobs for more-gettext-support blueprint https://review.openstack.org/197448 | 19:49 |
| *** jasondotstar has joined #openstack-infra | 19:50 | |
| AJaeger | mtreinish: shall I take the patch over? | 19:50 |
| mtreinish | AJaeger: sure go for it | 19:50 |
| *** cdent has quit IRC | 19:50 | |
| mtreinish | I've had a diminishing margin of interest in it as it increased from a 2 line change :) | 19:50 |
| *** irdkehn has joined #openstack-infra | 19:51 | |
| *** geoffarnold has joined #openstack-infra | 19:52 | |
| *** glauco has quit IRC | 19:52 | |
| *** kgiusti has left #openstack-infra | 19:54 | |
| openstackgerrit | Merged openstack-infra/system-config: Create grafana.o.o under -infra https://review.openstack.org/179208 | 19:54 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 19:54 |
| openstackgerrit | Merged openstack-infra/project-config: add project fuel-plugin-availability-zones https://review.openstack.org/199084 | 19:54 |
| AJaeger | mtreinish: hope this does the job ^ | 19:54 |
| mtreinish | AJaeger: cool, thanks | 19:55 |
| *** ig0r_ has quit IRC | 19:56 | |
| *** shashank_hegde has joined #openstack-infra | 19:56 | |
| zaro | pelix: is https://review.openstack.org/#/c/154316 no longer needed? | 19:57 |
| *** julim has joined #openstack-infra | 19:57 | |
| *** otter768 has joined #openstack-infra | 19:57 | |
| *** devkulkarni has quit IRC | 19:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add a gate job for Django-1.8 https://review.openstack.org/201467 | 19:57 |
| *** ivar-laz_ has quit IRC | 19:58 | |
| *** soren has joined #openstack-infra | 20:00 | |
| *** esmute_ has quit IRC | 20:00 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: gerrit/puppet-openstack: create puppet-manager-release group https://review.openstack.org/207596 | 20:01 |
| toabctl | dhellmann: the master tarballs for neutron-lbaas are outdated: http://tarballs.openstack.org/neutron-lbaas/ | 20:01 |
| EmilienM | AJaeger: ^ | 20:01 |
| toabctl | dhellmann: how to fix that? | 20:01 |
| annegentle | pabelanger: I think so? | 20:01 |
| AJaeger | EmilienM: I'll review, thanks | 20:01 |
| EmilienM | AJaeger: is the group automatically created? who is going to add crinkle and I in the group once created? | 20:01 |
| annegentle | pabelanger: looking | 20:01 |
| *** otter768 has quit IRC | 20:01 | |
| annegentle | pabelanger: I think I do | 20:02 |
| AJaeger | EmilienM: automatically created after merge - and you need to ask one of the friendly infra roots to add you and then you can add crinkle | 20:02 |
| toabctl | lifeless: oslo.concurrency has posix-ipc in it's requirements.txt . It should be posix_ipc . | 20:02 |
| EmilienM | AJaeger: cool thx again | 20:02 |
| annegentle | But they don't actually get uploaded to that web server without FTP access thorugh the infra account | 20:02 |
| lifeless | toabctl: ok. Uhm, why are you telling me? | 20:02 |
| toabctl | lifeless: I thought you are working on that things... | 20:03 |
| *** pc_m_AWAY has quit IRC | 20:03 | |
| toabctl | no? | 20:03 |
| nm_ | fungi clarkb: thanks | 20:03 |
| zaro | pelix: nevermind. i've commented. | 20:03 |
| dhellmann | toabctl: the job was broken, but the next time it runs it should update them /cc fungi, jeblair, clarkb | 20:03 |
| toabctl | dhellmann: thanks! | 20:03 |
| lifeless | toabctl: I'm not hand-crafting the requirements for all of openstack :) | 20:03 |
| docaedo | AJaeger: just wanted to say thanks for the quick and useful feedback on my apps-catalog-ui repo commit :) | 20:03 |
| toabctl | lifeless: the bot proposed the wron changes. | 20:04 |
| toabctl | lifeless: if you point me to the code of the bot, I can have a look.... | 20:04 |
| AJaeger | docaedo: you're welcome! | 20:04 |
| fungi | dhellmann: the requirements sync proposal job was broken? was it misnormalizing package names or something? | 20:04 |
| lifeless | toabctl: the updater code is in project-config - grep for 'update-requirements' | 20:04 |
| toabctl | lifeless: btw I thought you are the bot :-) | 20:04 |
| dhellmann | fungi: the tarball job | 20:04 |
| lifeless | toabctl: the update-requirements code itself is in openstack/requirements | 20:04 |
| lifeless | toabctl: and is data driven; the data is in the file global-requirements.txt there | 20:04 |
| *** sinval has joined #openstack-infra | 20:05 | |
| fungi | dhellmann: how did that create a bad requirements sync? | 20:05 |
| dhellmann | fungi: there's a post-commit job for building tarballs from branches, right? | 20:05 |
| lifeless | toabctl: which does have posix_ipc | 20:05 |
| dhellmann | fungi: I think we're talking about 2 different things | 20:05 |
| dhellmann | fungi: "<toabctl>dhellmann: the master tarballs for neutron-lbaas are outdated: http://tarballs.openstack.org/neutron-lbaas/" | 20:05 |
| *** tiswanso has joined #openstack-infra | 20:05 | |
| fungi | dhellmann: oh, sorry, yes. i thought you were answering the second question about the posix-ipc requirement name | 20:05 |
| lifeless | toabctl: I see that place it changed - commit 45ff5571380ea4fbd1cab6dca76bd05ebe59b317 | 20:06 |
| dhellmann | fungi: ah, no :-) | 20:06 |
| lifeless | toabctl: which was from the bot | 20:06 |
| lifeless | toabctl: so I think the bug will be in openstack/requirements | 20:06 |
| lifeless | toabctl: clone that, pip install into a virtual env and run update-requirements -h to get help | 20:07 |
| lifeless | something like update-requirements /path/to/oslo.concurrency will let you test it | 20:07 |
| lifeless | ok, I have to go do conferency stuff - sorry | 20:07 |
| toabctl | lifeless: ok. thanks a lot! | 20:07 |
| openstackgerrit | Brant Knudson proposed openstack-dev/pbr: Fix docs for environment markers in extras https://review.openstack.org/207600 | 20:08 |
| *** abdelwas has joined #openstack-infra | 20:08 | |
| lifeless | its likely that we're taking a normalised form where we should take the input form or something: OTOH thats not neessarily bad, and I'll want to talk to dstufft about pypi plans there | 20:08 |
| * lifeless goeth | 20:09 | |
| *** armax has joined #openstack-infra | 20:09 | |
| *** mwagner_lap has joined #openstack-infra | 20:09 | |
| *** armax has quit IRC | 20:09 | |
| *** mrmartin has quit IRC | 20:10 | |
| *** esmute_ has joined #openstack-infra | 20:10 | |
| openstackgerrit | John Trowbridge proposed openstack/diskimage-builder: Adds Ubuntu and Debian to ironic-python-agent Support-list https://review.openstack.org/198771 | 20:10 |
| openstackgerrit | John Trowbridge proposed openstack/diskimage-builder: Add ability to build ironic-python-agent ramdisk from packages https://review.openstack.org/207601 | 20:10 |
| *** e0ne has joined #openstack-infra | 20:11 | |
| *** mriedem1 is now known as mriedem_away | 20:12 | |
| *** annegentle has quit IRC | 20:13 | |
| AJaeger | fungi: your change to copy log files around is broken, see http://logs.openstack.org/68/199868/2/gate/project-config-gerrit/952499f/console.html | 20:13 |
| *** annegentle has joined #openstack-infra | 20:13 | |
| fungi | AJaeger: i thought it might be | 20:13 |
| fungi | AJaeger: hence my review comment | 20:14 |
| AJaeger | fungi: sorry, the log files change is not merged yet. | 20:14 |
| *** sabeen1 has joined #openstack-infra | 20:14 | |
| *** alivigni has quit IRC | 20:14 | |
| AJaeger | fungi: I fixed and reviewed the rename - but now I'm confused why the above fails. | 20:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-dashboard: This module is no longer supported https://review.openstack.org/207603 | 20:15 |
| fungi | toabctl: it looks like the log from the requirements update run which brought that change was http://logs.openstack.org/75/75aa400c7ea9f6e6694a472e183f59d0c8448f8b/post/propose-requirements-updates/bb2f2f6/console.html.gz#_2015-06-22_08_26_54_629 | 20:15 |
| *** baoli has quit IRC | 20:15 | |
| *** achanda has joined #openstack-infra | 20:16 | |
| *** openstackgerrit has quit IRC | 20:16 | |
| *** openstackgerrit has joined #openstack-infra | 20:17 | |
| AJaeger | fungi, jeblair, clarkb: I'm totally confused by the failure in https://review.openstack.org/199868, I don't understand why /usr/local/jenkins/slave_scripts/jenkins-sudo-grep.sh post failed in http://logs.openstack.org/68/199868/2/gate/project-config-gerrit/952499f/console.html. Any help appreciated | 20:19 |
| clarkb | jeblair not a huge fan of the explicit translate: true. maybe we can just add all projects? or is that confusing? | 20:20 |
| openstackgerrit | Merged openstack-infra/project-config: Add fuel-library channel https://review.openstack.org/200311 | 20:20 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: functions: allow to run cleanup_host outside of devstack context https://review.openstack.org/207606 | 20:21 |
| clarkb | jeblair and have preference to trigger locally like with gerrit | 20:21 |
| EmilienM | clarkb, crinkle ^ | 20:21 |
| *** esmute_ has quit IRC | 20:21 | |
| *** puranamr has joined #openstack-infra | 20:21 | |
| jeblair | clarkb: hrm, just at a guess, i think that might be confusing for potential translators who might translate things that then don't actually get used | 20:21 |
| AJaeger | clarkb: not all projects have translations - and having too many projects in zanata is confusing. We already have so many that we need to prioritize them | 20:21 |
| EmilienM | clarkb, crinkle: this is ugly but might work | 20:21 |
| *** glauco_ has joined #openstack-infra | 20:21 | |
| AJaeger | jeblair: +1 | 20:21 |
| *** esmute_ has joined #openstack-infra | 20:22 | |
| clarkb | ok | 20:22 |
| clarkb | then at least with setup that should mirror gerrit imo | 20:22 |
| jeblair | clarkb: i'm unsure about having that zuul-driven vs cron-driven | 20:22 |
| AJaeger | clarkb: we can do it differently but we should have a list of projects that have it enabled | 20:22 |
| clarkb | and go in jeepyb | 20:23 |
| clarkb | jeblair neither it should be puppet driven | 20:23 |
| jeblair | clarkb: or that | 20:23 |
| clarkb | its the existing pattern and we dont need two for this | 20:23 |
| jeblair | AJaeger: it sort of looks like that node was re-used; investigating. | 20:23 |
| AJaeger | clarkb: I'm fine with do it puppet driven - but then please explain StevenK how to do it... | 20:23 |
| openstackgerrit | Merged openstack-infra/project-config: Move gate-ironic-inspector-dsvm to (non-voting) check pipeline of ironic https://review.openstack.org/202682 | 20:23 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/devstack-gate: functions: allow to run cleanup_host outside of devstack context https://review.openstack.org/207606 | 20:24 |
| *** nm_ has quit IRC | 20:24 | |
| AJaeger | jeblair: if that's the case, then fungi's change https://review.openstack.org/203099 might have been run before... | 20:24 |
| AJaeger | and then it needs some more work | 20:24 |
| AJaeger | jeblair: thanks! | 20:24 |
| openstackgerrit | Merged openstack-infra/project-config: Add ALE Omniswitch Networking project to Openstack https://review.openstack.org/203477 | 20:25 |
| *** johnbelamaric has quit IRC | 20:26 | |
| *** puranamr has quit IRC | 20:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Add ironic-inspector to project list for python-ironic-inspector-client https://review.openstack.org/203629 | 20:28 |
| jeblair | oh _that's_ weird | 20:28 |
| *** maurosr has quit IRC | 20:28 | |
| jeblair | 2015-07-30 19:58:40.346 | > Jul 30 19:58:20 devstack-trusty-hpcloud-b1-4062737 sudo: jenkins : command not allowed ; TTY=unknown ; PWD=/home/jenkins/workspace/gate-tempest-dsvm-full-ceph ; USER=root ; COMMAND=/bin/cp /var/log/syslog /opt/stack/logs/syslog.txt | 20:28 |
| *** esmute_ has quit IRC | 20:28 | |
| jeblair | that log line showed up on this node: | 20:29 |
| jeblair | 2015-07-30 19:57:53.616 | Building remotely on bare-trusty-hpcloud-b2-4059105 (bare-trusty) in workspace /home/jenkins/workspace/project-config-gerrit | 20:29 |
| *** zz_zz_ja has quit IRC | 20:29 | |
| jeblair | note the completely different node names. | 20:29 |
| AJaeger | and bare vs devstack ;( | 20:29 |
| *** rfolco has quit IRC | 20:29 | |
| *** rfolco has joined #openstack-infra | 20:30 | |
| jeblair | so this looks less like an instance of "jenkins used a node twice" and more like an instance of "the same vm or disk image was used by two different nodes"; suggesting either a nodepool fault or openstack fault | 20:30 |
| * AJaeger will recheck the change | 20:30 | |
| clarkb | woah | 20:30 |
| *** samkirsch10 has joined #openstack-infra | 20:31 | |
| *** tonytan4ever has quit IRC | 20:31 | |
| *** puranamr has joined #openstack-infra | 20:32 | |
| jeblair | http://paste.openstack.org/show/406501/ | 20:32 |
| samkirsch10 | so… our team left go an employee, but before he left, he removed himself from our pypi package before he added anyone else. right now the only owner is openstackci. is there anyone here who has access to that and would be gracious to add me back to the package? | 20:32 |
| AJaeger | fungi: this is not at all related to your patch, you use mv not cp. Sorry fo rthe false alarm | 20:32 |
| AJaeger | samkirsch10: why do you need to be added there at all? | 20:33 |
| *** tiswanso has quit IRC | 20:33 | |
| samkirsch10 | our package is not currently in stackforge yet, so we need access to it to edit info | 20:33 |
| jeblair | clarkb: ah, there is an extra complication, in that nodepool thought it was a leaked instance and deleted it | 20:33 |
| fungi | jeblair: ooh, same ip address | 20:34 |
| *** cburroughs has quit IRC | 20:34 | |
| openstackgerrit | Devon Peters proposed openstack-infra/jenkins-job-builder: Add support for the RunDeck plugin https://review.openstack.org/205721 | 20:35 |
| AJaeger | samkirsch10: do you have an open review? I guess one of the infra roots like jeblair, clarkb or fungi can add you... | 20:35 |
| fungi | i wonder if it failed to delete, and then rackspace was confused and handed us a new ip address which was still in use by the stale worker | 20:35 |
| jeblair | fungi: s/rax/hp/ | 20:35 |
| fungi | er, right | 20:35 |
| fungi | blame where blame is due ;) | 20:35 |
| samkirsch10 | AJaeger: no review. just a former employee screwup. hoping someone can add me back as owner :) | 20:35 |
| fungi | that's a fip, so maybe they double-fip'd us? | 20:35 |
| fungi | double-dip-fip | 20:36 |
| openstackgerrit | Merged openstack-infra/project-config: Add fuel-octane project to stackforge https://review.openstack.org/203649 | 20:36 |
| samkirsch10 | AJaeger: https://pypi.python.org/pypi/python-monasca-events | 20:36 |
| *** puranamr has quit IRC | 20:36 | |
| openstackgerrit | David Medberry proposed openstack-infra/irc-meetings: Per IRC meeting owner, this is at 2000 https://review.openstack.org/207614 | 20:36 |
| *** tonytan4ever has joined #openstack-infra | 20:37 | |
| AJaeger | samkirsch10: I don't have permissions myself, hope somebody else will help you. | 20:37 |
| fungi | jeblair: it does certainly look a lot like hpcloud got its fip tracking mixed up | 20:37 |
| fungi | at least that's the simplest explanation | 20:37 |
| samkirsch10 | fungi: would you possibly be able to help? | 20:38 |
| *** baoli has joined #openstack-infra | 20:38 | |
| fungi | jeblair: the alternative is that somehow we looked up/recorded the wrong ip address in nodepool | 20:38 |
| fungi | samkirsch10: probably eventually. is it urgent? | 20:38 |
| *** baoli has quit IRC | 20:39 | |
| samkirsch10 | fungi: relatively urgent. | 20:39 |
| samkirsch10 | fungi: we've got new code that fixes some breakages and would like to push up | 20:39 |
| *** bauzas has quit IRC | 20:39 | |
| *** baoli has joined #openstack-infra | 20:39 | |
| *** annegentle has quit IRC | 20:40 | |
| * AJaeger waves good night... | 20:40 | |
| *** AJaeger has quit IRC | 20:41 | |
| *** Somay has joined #openstack-infra | 20:41 | |
| *** mjturek1 has left #openstack-infra | 20:41 | |
| fungi | samkirsch10: i don't see any sign that the python-monasca-events package has any release automation in our infrastructure at the moment. was it in the process of being imported to openstack-infra/project-config? | 20:43 |
| *** bauzas has joined #openstack-infra | 20:43 | |
| openstackgerrit | Khai Do proposed openstack-infra/jenkins-job-builder: Update sphinx to be compatible with readthedocs.org https://review.openstack.org/207616 | 20:43 |
| *** dguerri is now known as dguerri` | 20:43 | |
| samkirsch10 | fungi: openstackci was added as an owner for future proofing when we initiate the process | 20:43 |
| *** akshai has quit IRC | 20:44 | |
| fungi | samkirsch10: you want it updated so that the package index owner list is consistent with https://pypi.python.org/pypi/python-monascaclient ? | 20:44 |
| samkirsch10 | fungi: that would be awesome | 20:45 |
| *** jamespage_ has quit IRC | 20:45 | |
| *** bauzas has left #openstack-infra | 20:45 | |
| *** puranamr has joined #openstack-infra | 20:45 | |
| *** bswartz has quit IRC | 20:46 | |
| fungi | samkirsch10: done | 20:46 |
| samkirsch10 | fungi: you're awesome. saved the day! | 20:46 |
| *** openstackgerrit has quit IRC | 20:46 | |
| *** openstackgerrit has joined #openstack-infra | 20:47 | |
| *** ivar-lazzaro has joined #openstack-infra | 20:50 | |
| *** ivar-lazzaro has quit IRC | 20:50 | |
| openstackgerrit | Khai Do proposed stackforge/python-jenkins: Update sphinx to be compatible with readthedocs.org https://review.openstack.org/207618 | 20:51 |
| *** samkirsch10 has left #openstack-infra | 20:52 | |
| *** aysyd has quit IRC | 20:55 | |
| *** jasondotstar has quit IRC | 20:56 | |
| *** jamielennox|away is now known as jamielennox | 20:57 | |
| fungi | okay, jenkins01 is cleaned up and restarted. jenkins05 is still wrapping up a couple of long-running jobs. i'm putting jenkins06 in prepare for shutdown now | 20:57 |
| *** sinval has quit IRC | 20:58 | |
| *** woodster_ has quit IRC | 21:00 | |
| *** openstack has joined #openstack-infra | 21:10 | |
| *** MarkAtwood has quit IRC | 21:11 | |
| anteaya | rick_: additionally you have three gerrit accounts all using the same email address: http://lists.openstack.org/pipermail/third-party-announce/2015-July/000258.html | 21:11 |
| *** MarkAtwood has joined #openstack-infra | 21:11 | |
| *** esmute_ has joined #openstack-infra | 21:12 | |
| *** prad has quit IRC | 21:12 | |
| anteaya | rick_: solution, a) stop doing that, b) tell us which one of the three you want to keep, c) make future gerrit accounts with unique email addresses | 21:12 |
| *** tiswanso has quit IRC | 21:12 | |
| *** dkranz has quit IRC | 21:13 | |
| *** tlian has quit IRC | 21:13 | |
| *** Guest49146 has quit IRC | 21:14 | |
| *** marzif_ has quit IRC | 21:14 | |
| *** yamamoto has joined #openstack-infra | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| *** annegentle has joined #openstack-infra | 21:16 | |
| openstackgerrit | Thomas Bechtold proposed openstack/requirements: Use original package name when updating requirements https://review.openstack.org/207629 | 21:16 |
| *** zz_ja has joined #openstack-infra | 21:18 | |
| *** zul has joined #openstack-infra | 21:19 | |
| openstackgerrit | Merged openstack-infra/bindep: Add profile to the usage text https://review.openstack.org/203272 | 21:19 |
| *** MarkAtwood has quit IRC | 21:19 | |
| *** yamamoto has quit IRC | 21:19 | |
| *** maurosr has joined #openstack-infra | 21:19 | |
| openstackgerrit | Merged openstack-infra/system-config: Use valid cgitrc configuration key https://review.openstack.org/204316 | 21:19 |
| *** flepied1 has joined #openstack-infra | 21:20 | |
| Swanson | Is it safe to recheck a stable/kilo patch? | 21:20 |
| *** flepied has quit IRC | 21:20 | |
| sigmavirus24 | or start approving other patches? | 21:20 |
| *** arxcruz has quit IRC | 21:21 | |
| *** soren has quit IRC | 21:21 | |
| anteaya | dzamboni: I have no context for what you want from be based on your name and ping doesn't give me a lot to go on, perhaps post your question in the channel? | 21:22 |
| anteaya | s/be/me | 21:23 |
| *** achanda has quit IRC | 21:26 | |
| *** marzif_ has joined #openstack-infra | 21:26 | |
| *** bswartz has joined #openstack-infra | 21:27 | |
| *** geoffarnold has quit IRC | 21:29 | |
| openstackgerrit | Merged openstack-infra/project-config: Add stable compat jobs to trove-integration repo https://review.openstack.org/207055 | 21:30 |
| * anteaya goes to look for breakfast | 21:30 | |
| clarkb | I take it anteaya is not at home | 21:30 |
| clarkb | ok I think I am actually around the computer now :/ also its going to be really hot today so I may just melt into a puddle under the laptop | 21:31 |
| clarkb | anything I can help with re nodepool/jenkins/zuul? | 21:31 |
| *** marzif_ has quit IRC | 21:32 | |
| *** [HeOS] is now known as HeOS | 21:32 | |
| phschwartz | is it safe to set node: {node} where node is set in the project in jjb? | 21:32 |
| *** marzif_ has joined #openstack-infra | 21:34 | |
| clarkb | phschwartz: it is if you are happy with the default node choice for the project | 21:34 |
| clarkb | otherwise you will need to either specify it in the job-template directly or pass it to the job template when it is instantiated in the jobs list | 21:34 |
| *** baoli has quit IRC | 21:35 | |
| *** marzif_ has quit IRC | 21:35 | |
| clarkb | I am going to finish rereading the zuulv3 spec now in order to vote on it. Please let me know if I can be of assistance elsewhere | 21:35 |
| *** marzif_ has joined #openstack-infra | 21:36 | |
| *** maurosr has quit IRC | 21:36 | |
| *** zz_ja has quit IRC | 21:37 | |
| *** banix has quit IRC | 21:38 | |
| *** gordc_ is now known as gordc | 21:39 | |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 21:40 |
| *** achanda has joined #openstack-infra | 21:41 | |
| *** geoffarnold has joined #openstack-infra | 21:42 | |
| *** BrandonL08 has quit IRC | 21:42 | |
| *** BrandonL08 has joined #openstack-infra | 21:43 | |
| *** lykinsbd has quit IRC | 21:43 | |
| *** geoffarn_ has joined #openstack-infra | 21:43 | |
| fungi | jeblair: on your comment for 203099, we have a _lot_ of slave scripts which are calling testr indirectly under tox (often via pbr setup entrypoints like cover or build_sphinx) | 21:44 |
| fungi | jeblair: i can add yet a new script which does this and add it to the ~10 (according to git grep) scripts which fit those patterns | 21:45 |
| *** markvoelker_ has quit IRC | 21:46 | |
| *** geoffarnold has quit IRC | 21:47 | |
| jeblair | fungi: hrm, it sounds like something has gone terribly wrong. we're supposed to have a script that knows how to run testr in tox... | 21:47 |
| *** shashank_hegde has quit IRC | 21:48 | |
| *** jasondotstar has joined #openstack-infra | 21:48 | |
| jeblair | fungi: the one i'm thinking of seems to be called 'run-tox' these days | 21:49 |
| *** annegentle has quit IRC | 21:49 | |
| fungi | jeblair: yep, run-tox.sh does the unit tests, but we have other scripts for things like pep8 | 21:50 |
| jeblair | jogo: are you reviewing multinode test changes these days? ianw wanted to see an ack from you on https://review.openstack.org/207191 but i don't know if you're into that sort of thing. | 21:50 |
| fungi | jeblair: oh, though i guess those don't end up calling testr | 21:50 |
| jeblair | fungi: yeah, that was what i was thinking | 21:50 |
| fungi | anyway, yes putting it in run-tox.sh will at least get the majority (i think coverage may be the main outlier) | 21:51 |
| clarkb | we can probably change coverage over to it | 21:51 |
| *** ig0r__ has quit IRC | 21:51 | |
| clarkb | trying to remember if run-cover does anything special anymore (it did once upon a time but now pbr knows how to do it for us) | 21:51 |
| *** thorst1 has joined #openstack-infra | 21:51 | |
| jeblair | it looks like there's _some_ references to testr in a few jobs, but nothing that makes me change my mind | 21:52 |
| lifeless | fungi: build_sphinx calls testr? | 21:52 |
| fungi | lifeless: nope, i was spacing | 21:52 |
| fungi | just coverage i think | 21:52 |
| jeblair | wow, run-cover does nothing | 21:52 |
| *** hdd has joined #openstack-infra | 21:53 | |
| *** marzif_ has quit IRC | 21:54 | |
| *** shashank_hegde has joined #openstack-infra | 21:55 | |
| jogo | jeblair: I haven't tracked multinode for a while now | 21:55 |
| *** skylerberg has joined #openstack-infra | 21:55 | |
| jogo | but is -full really working? | 21:55 |
| jogo | clarkb: ^ | 21:56 |
| greghaynes | it was last I checked | 21:56 |
| clarkb | jogo: from what I can see yes (I relly need to stress that full/smoke was basically the same fail rate except for a tempest bug that was fixed) | 21:56 |
| jogo | greghaynes: link to graphite? | 21:56 |
| greghaynes | jul 19 I had a pass | 21:56 |
| clarkb | jogo: see above smoke was not any more reliable | 21:57 |
| greghaynes | heh, I dont have fancy graphite links | 21:57 |
| clarkb | (this is why I was frustrated we did that switch) | 21:57 |
| clarkb | the issues were related to test VM setup which affected smoke and full equally | 21:57 |
| jogo | greghaynes: looks like the job names changed or something, because I ahve no data http://jogo.github.io/gate/multinode | 21:57 |
| jogo | clarkb: what about the nova migration tests that failed connectivity? | 21:57 |
| jogo | after a instance migration | 21:58 |
| openstackgerrit | Merged openstack-infra/project-config: Only run constraints proposal for master https://review.openstack.org/205292 | 21:58 |
| clarkb | jogo: that was a bug in devstack gate | 21:58 |
| *** otter768 has joined #openstack-infra | 21:58 | |
| clarkb | my point is we should've fixed full | 21:58 |
| clarkb | we got distracted instead and now I am trying to get back to where we shoul dbe | 21:58 |
| jogo | clarkb: so graphite graph? | 21:58 |
| *** gordc has quit IRC | 21:59 | |
| *** abregman is now known as abregman|ZzZzZ | 21:59 | |
| lifeless | I've added a comment to that review | 21:59 |
| lifeless | subunit files are not text. | 21:59 |
| clarkb | jogo: I think if you s/check/gate/ your graphs will work | 21:59 |
| greghaynes | gate-tempest-dsvm-neutron-multinode-full | 22:00 |
| greghaynes | thats the job last I saw | 22:00 |
| lifeless | Neither v1 nor v2, and treating them as text will break their encoding due to the normalisation that occurs. | 22:00 |
| fungi | lifeless: sorry, now i realize how i just confused myself. i wanted to switch tox logs to text | 22:00 |
| lifeless | of course, if these are not about the subunit files, thats cool. | 22:00 |
| fungi | jeblair: ^ | 22:00 |
| *** annegentle has joined #openstack-infra | 22:00 | |
| fungi | i said testr in error and then went down the wrong rabbit hole | 22:00 |
| greghaynes | er, not neutron, but you get the idea ;) | 22:01 |
| openstackgerrit | Merged openstack-infra/project-config: Additional tweaks to anvil rpm/venv building jobs https://review.openstack.org/205633 | 22:01 |
| fungi | jeblair: lifeless: so ideally we do something that makes all the tox logs we collect viewable in-browser, which i attempted to centralize by doing it where we collect the logs from tox-based jobs. we call tox in lots of scripts though (not just the ones where we're using testr) | 22:02 |
| *** e0ne has quit IRC | 22:02 | |
| *** otter768 has quit IRC | 22:03 | |
| jogo | clarkb: yup | 22:03 |
| clarkb | jogo: ya that seems to work though teh resulting grpah is suspiciously showing a 0% fail rate | 22:03 |
| jogo | http://graphite.openstack.org/render/?from=00%3A00_20150727&yMax=100&fgcolor=000000&title=Check%20Failure%20Rates%20(36%20hours)&_t=0.8301145927980542&height=308&bgcolor=ffffff&width=586&yMin=0&until=23%3A59_20150730&showTarget=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.check-tempest-dsvm-multinode-full.FAILURE%2Csum(stats.zuul.pipeline.check.job.check-tempest-dsvm-multinode-fu | 22:03 |
| jogo | ll.%7BSUCCESS%2CFAILURE%7D))%2C%275hours%27)%2C%20%27check-tempest-dsvm-multinode-full%27)%2C%27brown%27)&_salt=1438293792.211&target=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-full.FAILURE%2Csum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-full.%7BSUCCESS%2CFAILURE%7D))%2C%275hours%27)%2C%20%27gate-tempest-dsvm-full%27)%2C%27orange%27)&target=color(alias(m | 22:03 |
| jogo | ovingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-multinode-full.FAILURE%2Csum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-multinode-full.%7BSUCCESS%2CFAILURE%7D))%2C%275hours%27)%2C%20%27gate-tempest-dsvm-multinode-full%27)%2C%27brown%27) | 22:03 |
| jeblair | jogo: tinyurl will make that more useful to people next time :) | 22:04 |
| jogo | jeblair: forgot how long that was :( | 22:05 |
| jogo | http://tinyurl.com/prabf45 | 22:05 |
| *** sdake has quit IRC | 22:05 | |
| jeblair | fungi: yeah, a lot of those use run-tox.sh though | 22:05 |
| *** sdake has joined #openstack-infra | 22:05 | |
| jeblair | fungi: inasmuch as the 'tox' builder macro runs run-tox, and i think most of them use that macro | 22:06 |
| *** ybathia has quit IRC | 22:06 | |
| *** ybathia_ has joined #openstack-infra | 22:06 | |
| fungi | jeblair: except pep8, docs, coverage, tarballs... | 22:06 |
| *** ybathia_ has quit IRC | 22:07 | |
| fungi | but i agree that recombining all those (or at least extracting the tox running out of them and centralizing that) is likely a good first step | 22:08 |
| jeblair | fungi: aren't you just adding it to the test-results publisher? | 22:08 |
| jogo | clarkb: we still have nova-computes hanging | 22:08 |
| *** markvoelker has joined #openstack-infra | 22:08 | |
| jogo | http://logs.openstack.org/92/176892/13/check/gate-tempest-dsvm-multinode-full/ac85409/logs/subnode-2/screen-n-cpu.txt.gz | 22:08 |
| *** ybathia has joined #openstack-infra | 22:09 | |
| clarkb | jogo: but how does smoke affect that? | 22:09 |
| fungi | jeblair: oh, hrm... i bet we're not collecting tox logs from anything else, huh? that would explain why sometimes when i go to debug tox-related issues we haven't collected them | 22:09 |
| clarkb | jogo: my point is lets test the thing that actually finds bugs then fix the bugs | 22:09 |
| clarkb | jogo: because what we did previously was bandaid hoping it would fix things (and didn't) | 22:09 |
| *** angela-s has joined #openstack-infra | 22:09 | |
| jogo | clarkb: smoke didn't hit that AFAIK | 22:09 |
| *** amitgandhinz has quit IRC | 22:10 | |
| clarkb | jogo: that tells me we should either be running full in order to catch it or we have gotten lucky with smoke | 22:10 |
| *** sdake has quit IRC | 22:10 | |
| clarkb | jogo: but by not running it we don't get the data we need to fix it | 22:11 |
| clarkb | *by not running full | 22:11 |
| *** sdake has joined #openstack-infra | 22:11 | |
| jogo | clarkb: smoke doesn't do any migration tests AFAIK | 22:11 |
| jogo | clarkb: we already run tempest multinode full in few places (nova in particular) | 22:12 |
| jogo | so we have the data | 22:12 |
| jogo | just no one to fix it | 22:12 |
| clarkb | I have been fixing the bugs or working with others to fix them the last couple weeks... | 22:12 |
| clarkb | fixed a tempest bug, fixed several infra issues etc | 22:12 |
| *** angela-s has left #openstack-infra | 22:13 | |
| jogo | clarkb: nice | 22:13 |
| clarkb | so back to my point | 22:13 |
| jogo | my take is we don't actually need more data to get the job stable | 22:13 |
| clarkb | using smoke is a bandaid that didn't actually help us when things were majorly broken | 22:13 |
| clarkb | lets go back to using full so we can find the issues and fix them | 22:13 |
| jogo | clarkb: sure go for it, if you are maintaining the job, your call | 22:14 |
| jogo | the job looks like it is still non-voting | 22:14 |
| clarkb | it is, all this does is s/smoke/full/ | 22:14 |
| clarkb | because smoke did not add anything and actively hampered our efforts here | 22:14 |
| *** jamielennox is now known as jamielennox|away | 22:15 | |
| *** bpokorny has joined #openstack-infra | 22:15 | |
| jogo | clarkb: well when I added the smoke job in initially (not sure if that what this is), it was there to see if we can use multinode for grenade | 22:16 |
| jogo | where we use smoke | 22:16 |
| clarkb | jogo: I know, but full is a super set of smoke | 22:16 |
| clarkb | jogo: and the jobs were all failing at the time due to infra issues that needed to be fixed | 22:16 |
| jogo | clarkb: it was originally added as a temporary measure | 22:16 |
| clarkb | then why are we even discussing untemporarying it? | 22:16 |
| jogo | clarkb: don't know | 22:17 |
| jogo | clarkb: if you want to run full to get better data on what is failing to make it stable, great | 22:17 |
| openstackgerrit | Kyle Mestery proposed openstack-infra/project-config: vpnaas: Turn on python3 jobs https://review.openstack.org/207650 | 22:17 |
| jogo | if you already think you have enough data (nova is pretty active) then I had a patch to drop them | 22:17 |
| clarkb | jogo: we don't want to drop them though | 22:18 |
| clarkb | we just want to replace smoke with full | 22:18 |
| clarkb | which is what the chagne under discussion does | 22:18 |
| jogo | clarkb: and I am saying go for it | 22:18 |
| clarkb | cool | 22:18 |
| jogo | I think its premature which is why I don't want to +1 it. but don't want to -1, so +0 | 22:18 |
| clarkb | can you clarify on that? | 22:19 |
| clarkb | what specifically is premature? running tempest full non voting? | 22:19 |
| *** yamamoto has joined #openstack-infra | 22:19 | |
| jogo | clarkb: because we already run it on nova which gives us a good dataset of what issues remain | 22:19 |
| jogo | clarkb: so running full in more places won't give us much new information | 22:20 |
| clarkb | we don't/didn't run neutron though | 22:20 |
| clarkb | so a big part of this is treat neutron as equal | 22:20 |
| clarkb | and fix the bugs | 22:20 |
| jogo | clarkb: true, I haven't looked at the neutron job in a long long time | 22:21 |
| jogo | last I checked it never worked (outdated information) | 22:21 |
| jogo | clarkb: because it was the whole DVR mes | 22:21 |
| clarkb | it works now without dvr | 22:21 |
| jogo | s | 22:21 |
| zigo | dims: Do you believe that "oslo.reports library" is fine for a package long description? | 22:21 |
| clarkb | and dvr is being worked separately as that was the direction neutron wanted togo | 22:21 |
| jogo | clarkb: anyway, I am just happy someone is running with this | 22:21 |
| jogo | excellent | 22:21 |
| *** thorst1 has quit IRC | 22:22 | |
| zigo | dims: Said in a nicer way: WTF is this long description, who's the authors so I can kill them? :D | 22:22 |
| jogo | clarkb: I am really excited about seeing this getting worked on, I can't stress that enough | 22:22 |
| clarkb | or at least it appears to work so now lets throw a little load at it and find out if that holds up | 22:22 |
| jogo | clarkb: ++, I stopped caring about the neutron job long ago, since the whole DVR mess | 22:23 |
| SpamapS | zigo: something got lost in translation there.. not so sure you used the word "nice" in the correct way. ;) | 22:24 |
| jeblair | zigo: less with the killing and more with the being polite, please :) | 22:24 |
| zigo | jeblair: I wasn't being serious ... | 22:24 |
| *** yamamoto has quit IRC | 22:24 | |
| clarkb | jogo: so you are good with removing the two extra smoke jobs and replacing them with two full jobs? essentially just taking the testing we were doing before and running the additonal tempest tests? | 22:25 |
| clarkb | jogo: they are still non voting | 22:25 |
| jogo | clarkb: yes, DO IT | 22:25 |
| clarkb | this frees up a few nodes by reducing the redundant job count and gets us neutron + tempest full on multnode | 22:25 |
| clarkb | awesome | 22:25 |
| jogo | clarkb: oh nice I didn't realize this removes some from redundant jobs | 22:27 |
| clarkb | jogo: we were running full and smoke in some cases | 22:27 |
| clarkb | (but nt full + neutron just nova net) | 22:27 |
| clarkb | so we go from 3 jobs to 2 | 22:28 |
| *** Hal has joined #openstack-infra | 22:29 | |
| *** Hal is now known as Guest47866 | 22:29 | |
| bpokorny | Hi all, do I need to "reverify" this review to get it back in the zuul queue? https://review.openstack.org/#/c/206765/ | 22:30 |
| bpokorny | I saw the ML thread on some recent issues, so just want to make sure that's ok to do now. | 22:31 |
| *** markvoelker_ has joined #openstack-infra | 22:32 | |
| *** BrandonL08 has quit IRC | 22:33 | |
| *** geoffarnold has joined #openstack-infra | 22:34 | |
| clarkb | bpokorny: I think right now things are moving but doing so slowly and we expect zuul to get stuck again but with additional logging which should lalow for us to fix it | 22:35 |
| *** markvoelker has quit IRC | 22:35 | |
| clarkb | bpokorny: so you can reverify but it may need to be done again depending on how slowly this goes | 22:35 |
| *** ivar-lazzaro has joined #openstack-infra | 22:35 | |
| bpokorny | Thanks, clarkb. I'll do that for now and check on it after a while. | 22:36 |
| *** geoffarn_ has quit IRC | 22:37 | |
| *** eharney has quit IRC | 22:38 | |
| *** Sukhdev_ has joined #openstack-infra | 22:39 | |
| *** FallenPegasus has joined #openstack-infra | 22:40 | |
| *** FallenPegasus is now known as MarkAtwood | 22:40 | |
| reed | Shrews, have you ever seen this for shade? http://paste.openstack.org/show/406510/ | 22:41 |
| *** jsavak has quit IRC | 22:42 | |
| *** dimtruck is now known as zz_dimtruck | 22:43 | |
| Shrews | reed: no. something wrong with the designate client install. try doing 'pip install -e' instead to see if that works | 22:43 |
| *** annegentle has quit IRC | 22:44 | |
| *** harlowja has quit IRC | 22:44 | |
| *** jaosorior has quit IRC | 22:44 | |
| *** harlowja has joined #openstack-infra | 22:44 | |
| reed | Shrews, I was trying to install the latest version from trunk, with the fixes for DreamHost | 22:45 |
| *** jswarren has quit IRC | 22:45 | |
| clarkb | jeblair: fungi is zuul stuck now? | 22:46 |
| clarkb | looking at the status I don't think it is processing events | 22:46 |
| fungi | clarkb: dunno, eating dinner and trying to finish cleaning up jenkins06 | 22:46 |
| jeblair | clarkb: i believe it has encountered the error that will make it stuck, i was just checking on that | 22:46 |
| Shrews | reed: works fine for me. not a shade issue. more likely a setuptools problem or something. i always do 'pip install -e .' | 22:47 |
| jeblair | so i'm fairly certain it will be stuck, i'm not sure it is now. | 22:47 |
| clarkb | jeblair: looking at the debug log it is still active | 22:47 |
| clarkb | so likely not stuck stuck | 22:47 |
| jeblair | yep, not stuck | 22:47 |
| fungi | okay, jenkins06 is cleaned and on its way back up | 22:47 |
| jeblair | 2015-07-30 22:13:40,386 ERROR zuul.Server: Reconfiguration failed: | 22:47 |
| jesusaurus | jeblair: what is the error that makes it stuck? | 22:47 |
| *** abdelwas has quit IRC | 22:48 | |
| *** hdd has quit IRC | 22:48 | |
| jeblair | but there's the start of our new traceback! :) | 22:48 |
| reed | Shrews, good to know... in fact it seems to be running now, thanks | 22:48 |
| jeblair | (the nature of the bug is that it happens, and then at some point in the future, it causes zuul to get stuck. so our current run could continue for another 14 hours without us noticing) | 22:48 |
| jesusaurus | huh | 22:49 |
| jeblair | and here's the actual traceback: http://paste.openstack.org/show/406511/ | 22:49 |
| reed | Shrews, thanks, pip install -e works | 22:49 |
| jeblair | (lines are wrong due to the revert, but should be able to reverse that) | 22:50 |
| clarkb | jeblair: that looks like possibly related to the fix for the we didn't reset state on builds when reconfiguring bug? | 22:51 |
| jeblair | clarkb: yep; traded one stuck for another stuck :/ | 22:51 |
| jeblair | (a little stuck for a big stuck?) | 22:51 |
| jeblair | anyway, i'll go test&patch now | 22:51 |
| *** MarkAtwood has quit IRC | 22:52 | |
| Daviey | Hey, if someone wants to do a drive by contribution.. Are we allowed to sponsor a git cherry-pick on their behalf to gerrit? (They haven't signed the Contributor agreement for example) | 22:53 |
| *** Guest47866 has quit IRC | 22:57 | |
| fungi | Daviey: that's a question for legal counsel | 22:58 |
| Daviey | fungi: Has it never come up before? | 22:58 |
| jeblair | 207135,3 is the change that's going to get zuul stuck when it reports | 22:59 |
| fungi | Daviey: i am not a lawyer, but i believe the answer is that the executive director of the foundation has the option of determining whether a contribution is trivial enough to not require explicit licensing under the terms of the openstack contributor license agreement | 22:59 |
| *** boris-42 has quit IRC | 23:00 | |
| fungi | Daviey: the question has definitely come up in the past, usually as a hypothetical, gets directed to the legal-discuss@l.o.o ml and the answer is usually vague and "it depends" | 23:00 |
| Daviey | fungi: FML | 23:00 |
| Daviey | (thanks) | 23:01 |
| *** sergmelikyan has quit IRC | 23:06 | |
| *** sabeen1 has quit IRC | 23:06 | |
| *** esmute_ has quit IRC | 23:07 | |
| *** esmute_ has joined #openstack-infra | 23:08 | |
| *** tiswanso has joined #openstack-infra | 23:09 | |
| *** markvoelker_ has quit IRC | 23:14 | |
| *** cburroughs has joined #openstack-infra | 23:15 | |
| *** gildub has joined #openstack-infra | 23:16 | |
| *** gildub_ has joined #openstack-infra | 23:16 | |
| *** ashleighfarnham has quit IRC | 23:16 | |
| *** rossella_s has quit IRC | 23:16 | |
| *** rossella_s has joined #openstack-infra | 23:16 | |
| *** glauco_ has quit IRC | 23:18 | |
| *** tqtran has joined #openstack-infra | 23:18 | |
| *** romcheg has quit IRC | 23:20 | |
| *** cburroughs has quit IRC | 23:20 | |
| *** tiswanso has quit IRC | 23:21 | |
| *** zz_dimtruck is now known as dimtruck | 23:22 | |
| reed | Shrews, progress :) shade creates the VM without complaining now but still fails if I try to attach a public IPv4 to it (just FYI, one of DreamHost people is investigating, too http://paste.openstack.org/show/406515/) | 23:25 |
| reed | Shrews, the good thing is that the floating IP is attached correctly anyway :) | 23:26 |
| *** shashank_hegde has quit IRC | 23:27 | |
| *** romcheg has joined #openstack-infra | 23:29 | |
| *** shashank_hegde has joined #openstack-infra | 23:30 | |
| EmilienM | ianw: clarkb suggested me to use devstack-gate | 23:30 |
| EmilienM | ianw: I'm really confused | 23:30 |
| reed | Shrews, what's the correct way to pass userdata to shade? i'm using http://paste.openstack.org/ but doesnt's seem to create anything | 23:30 |
| *** esmute_ has quit IRC | 23:30 | |
| *** dims_ has joined #openstack-infra | 23:31 | |
| *** dims_ has quit IRC | 23:31 | |
| dstanek | anyone know who owns the release for bashate? i think an issue i ran across is fixed in master, but the last release was some time ago | 23:34 |
| *** akraminsky has quit IRC | 23:34 | |
| *** esmute_ has joined #openstack-infra | 23:34 | |
| *** dims has quit IRC | 23:35 | |
| jhesketh | jeblair, clarkb, fungi: looks like we're hitting the same problem again | 23:35 |
| *** weshay has quit IRC | 23:36 | |
| ianw | EmilienM: well, we can have different opinions. if clarkb and someone else want to sign-off on it, that's fine. i think that we should abstrct a bit further from devstack-gate | 23:36 |
| ianw | dstanek: krotscheck, i think... | 23:36 |
| EmilienM | ianw: it's just boring to implement 5 different methods after reviews/feedback from others and now rejected again. I can't spend more time of having our logs | 23:37 |
| EmilienM | just to do a cp | 23:38 |
| dstanek | ianw: thx. i'll check in with krotscheck | 23:38 |
| *** armax has joined #openstack-infra | 23:38 | |
| *** armax has quit IRC | 23:39 | |
| *** jaypipes has quit IRC | 23:39 | |
| anteaya | <- pyconau, monty is onstage talking about stanislavski | 23:40 |
| *** esmute_ has quit IRC | 23:40 | |
| krotscheck | ianw: bashate? | 23:40 |
| krotscheck | dstanek: I don't think I'm that person | 23:40 |
| *** esmute_ has joined #openstack-infra | 23:41 | |
| crinkle | ianw: where do you propose to put the scp publisher? if we put it http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/puppet-module-jobs.yaml#n53 we don't have a way (or don't know of a way) to say which logs we need, since different puppet modules may need different logs | 23:42 |
| crinkle | which is why the devstack-gate function was suggested | 23:42 |
| *** marcusvrn has quit IRC | 23:43 | |
| ianw | dstanek / krotscheck : you're right, it was mrodden | 23:44 |
| *** doug-fish has quit IRC | 23:44 | |
| anteaya | can someone disable gerrit account id 15670 at thingee's request: http://paste.openstack.org/show/406518/ | 23:44 |
| krotscheck | tanke :) | 23:44 |
| dstanek | ianw, krotscheck: then i'll bug mrodden :-) thanks for the pointer | 23:45 |
| *** doug-fish has joined #openstack-infra | 23:45 | |
| ianw | dstanek: i think something has changed and all devstack-core can push a signed tag | 23:45 |
| *** vipul has quit IRC | 23:46 | |
| ianw | i'd just be really really careful because if we push a release and it starts rejecting changes, the gate can get really backed up | 23:46 |
| *** vipul has joined #openstack-infra | 23:46 | |
| ianw | and it's already pretty ad | 23:46 |
| ianw | bad | 23:46 |
| *** armax has joined #openstack-infra | 23:47 | |
| *** esmute_ has quit IRC | 23:47 | |
| *** esmute_ has joined #openstack-infra | 23:48 | |
| dstanek | yeah, i'm wondering if that's why we haven't pushed it yet | 23:49 |
| dstanek | although we can cut a release and not update g-r until we're pretty sure | 23:49 |
| clarkb | ianw mostly I didn't want to reinvent the get all associated logs from openstack from host problem | 23:50 |
| *** puranamr has quit IRC | 23:50 | |
| clarkb | ianw: devstack gate knows how to get mysql/postgres/rabbit/libvirt/keystone/nova/neutron/etc | 23:50 |
| reed | shade is magical | 23:50 |
| anteaya | clarkb: are you able to disable gerrit account 15670 for me? | 23:51 |
| reed | I was using it wrong :) | 23:51 |
| clarkb | anteaya: ya can do | 23:51 |
| anteaya | clarkb: thanks | 23:51 |
| jhesketh | clarkb: ping | 23:51 |
| reed | `auto_ip=True` really means assign automatically a public IP, that's it | 23:51 |
| clarkb | jhesketh: pong | 23:51 |
| StevenK | clarkb: There's a bunch of code in project-config, like ZanataRestService, which means moving the code wholesale has some issues | 23:52 |
| jhesketh | clarkb: looks like the bug is hitting us again | 23:52 |
| * jhesketh just finished reading backlog | 23:52 | |
| *** ZZelle_ has quit IRC | 23:52 | |
| ianw | EmilienM / clarkb : sure ... but isn't the core problem that puppet-keystone wants to get the apache and keystone log? | 23:52 |
| jhesketh | very sorry that my code seems to have caused all this :-( | 23:52 |
| clarkb | jhesketh: I think jeblair was working on it | 23:52 |
| clarkb | and its possible the fix for not reseting state caused this | 23:52 |
| clarkb | or is related | 23:52 |
| jhesketh | clarkb: which fix is that | 23:53 |
| clarkb | jhesketh: let me dig it up | 23:53 |
| EmilienM | ianw: this is not a puppet-keystone problem, but an infrastructure thing | 23:53 |
| ianw | dstanek: yeah, i think though because it's just used from tox, it bypasses all the g-r? | 23:53 |
| clarkb | anteaya: done | 23:53 |
| openstackgerrit | Michael Johnson proposed openstack/requirements: Add pyroute2 to global-requirements.txt https://review.openstack.org/207671 | 23:53 |
| anteaya | clarkb: thank you | 23:54 |
| jhesketh | clarkb: the part I do find odd, is that if any/all is broken it should only be broken in the sense that some things get queued that shouldn't and some things that should be queued aren't | 23:54 |
| EmilienM | ianw, clarkb: I could have built my script in our puppet repos and the problem would have been solved But it's duplicating what d-g is doing, clarkb is right I think | 23:54 |
| jhesketh | whereas the actually error appears to be more due to some kind of race in re-queing ,but I guess that could be related if the match conditions are wrong | 23:54 |
| *** patrickeast has quit IRC | 23:54 | |
| clarkb | ianw: the core problem is that openstack has a ton of related logs that we need to get | 23:55 |
| clarkb | ianw: and openstack doesn't write them consistently | 23:55 |
| *** dannywilson has quit IRC | 23:55 | |
| clarkb | jhesketh: 6bc782de75be6720aa31843e64f4cb57bfe01de9 that one I think | 23:56 |
| *** nelsnelson has quit IRC | 23:56 | |
| jhesketh | clarkb: which project sorry? | 23:56 |
| SpamapS | clarkb: you say that as if "OpenStack" is a single thing. ;) | 23:56 |
| clarkb | jhesketh: zuul | 23:56 |
| clarkb | SpamapS: well thats part of the problem | 23:56 |
| SpamapS | thats how all micro service architectures work | 23:56 |
| clarkb | SpamapS: but OpenStack projects could be told by something like the TC to wrote all your logs to /var/log/openstack | 23:57 |
| clarkb | and put all your config in /etc/openstack by default | 23:57 |
| clarkb | then you can just tar and scp | 23:57 |
| SpamapS | If you can get them to be 100% ops homogeneous you probably aren't isolating them all from eachother enough. :) | 23:57 |
| clarkb | as is half of them write to /var/log/foo others do syslog only | 23:57 |
| clarkb | some write to console | 23:57 |
| *** salv-orlando has quit IRC | 23:57 | |
| anteaya | clarkb: for reference: http://lists.openstack.org/pipermail/third-party-announce/2015-July/000259.html | 23:57 |
| EmilienM | clarkb++ | 23:57 |
| dstanek | ianw: no, it shouldn't. a project has to specify it in there requirements.txt or tox.ini (depending on how they are implemented) | 23:57 |
| clarkb | then you get into upstart vs systemd handling of console/syslog | 23:57 |
| clarkb | and now things start exploding into ridiculousness | 23:58 |
| jhesketh | clarkb: ah, that seems more likely to be related, but I'm only guessing | 23:58 |
| dstanek | ianw: the only problem i can think of is a project using an unbounded version to g-r would have to make sure that doesn't happen before a release | 23:58 |
| jhesketh | and sadly I'm running a conference this weekend so I'm flat out in terms of helping :-( | 23:58 |
| clarkb | jhesketh: it sounds like jeblair is on it | 23:58 |
| jhesketh | :-) | 23:59 |
| *** otter768 has joined #openstack-infra | 23:59 | |
| ianw | dstanek: which is devstack itself right -> https://git.openstack.org/cgit/openstack-dev/devstack/tree/tox.ini#n11 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!