| *** hamzy has joined #openstack-third-party-ci | 00:00 | |
| *** Apoorva has quit IRC | 00:03 | |
| *** devlaps has quit IRC | 00:37 | |
| *** devlaps has joined #openstack-third-party-ci | 00:38 | |
| *** Apoorva has joined #openstack-third-party-ci | 01:02 | |
| *** Apoorva has quit IRC | 01:02 | |
| *** Apoorva has joined #openstack-third-party-ci | 01:04 | |
| *** Apoorva has quit IRC | 01:04 | |
| *** viks has quit IRC | 02:06 | |
| *** Apoorva has joined #openstack-third-party-ci | 02:20 | |
| *** Apoorva has quit IRC | 02:34 | |
| *** rajinir has quit IRC | 03:46 | |
| *** devlaps has quit IRC | 03:58 | |
| *** watanabe_isao has joined #openstack-third-party-ci | 04:22 | |
| *** krtaylor has quit IRC | 05:16 | |
| *** krtaylor has joined #openstack-third-party-ci | 05:28 | |
| *** fdegir has quit IRC | 05:35 | |
| *** fdegir has joined #openstack-third-party-ci | 05:35 | |
| *** eandersson has joined #openstack-third-party-ci | 09:22 | |
| *** eandersson has quit IRC | 09:28 | |
| *** s-shiono has quit IRC | 10:23 | |
| *** rfolco has joined #openstack-third-party-ci | 11:06 | |
| *** stakeda has quit IRC | 12:04 | |
| *** stakeda has joined #openstack-third-party-ci | 12:05 | |
| *** watanabe_isao has quit IRC | 13:28 | |
| *** isaacb has joined #openstack-third-party-ci | 13:57 | |
| isaacb | mmedvede: here? | 13:58 |
|---|---|---|
| mmedvede | isaacb: yes | 13:58 |
| isaacb | Is there a way to queue jos in jenkins? | 13:59 |
| isaacb | *jobs* | 14:00 |
| mmedvede | yes, one of the things zuul does is queue jobs | 14:00 |
| mmedvede | do you use zuul? | 14:01 |
| isaacb | sure. | 14:01 |
| mmedvede | isaacb: so are you asking how to manually add a job to zuul queue? | 14:02 |
| isaacb | is there any parameter in zuul to configure what should be the queue for a single job? | 14:02 |
| mmedvede | zuul uses layout config file to figure out what which job to run for each patch | 14:03 |
| mmedvede | isaacb: for example, https://github.com/openstack-infra/project-config-example/blob/master/zuul/layout.yaml | 14:05 |
| isaacb | mmedvede: I know. suppose you have a provider in nodepool that can hold 10 VMs | 14:05 |
| isaacb | mmedvede: if you have more than 10 commits in short period of time how you can queue them? | 14:06 |
| mmedvede | zuul holds the queue | 14:06 |
| mmedvede | well, technically, zuul uses gearman server, which in turn hold the job queue | 14:07 |
| isaacb | mmedvede: the problem is that gearman triggers a job in jenkins. | 14:08 |
| isaacb | mmedvede: jenkins does not have enough slaves, so he tries to reuse a slave | 14:09 |
| isaacb | mmedvede: which cause failures | 14:09 |
| mmedvede | isaacb: I see | 14:09 |
| isaacb | mmedvede: is there a way to queue the triggers to jenkins? | 14:11 |
| mmedvede | yes, that is what zuul should normally do. It seems it does not happen in your setup for whatever reason | 14:11 |
| mmedvede | this is how it normally happens for zuul/nodepool/jenkins setup: | 14:12 |
| mmedvede | zuul has a patch come in, and adds jobs it into appropriate queues. Nodepool boots VMs according to demand, and registers each VM with jenkins | 14:13 |
| mmedvede | then jenkins registers workers with the queue. You should have one gearman worker registered per VM | 14:14 |
| isaacb | mmedvede: how zuul knows the max capacity of nodepool? | 14:15 |
| mmedvede | when there are more jobs than workers (i.e. more jobs than VMs to run them), jobs should just sit on gearman queue until more workers are available | 14:15 |
| mmedvede | isaacb: zuul does not know max capacity | 14:15 |
| mmedvede | isaacb: zuul only knows about gearman server | 14:17 |
| mmedvede | sidenote: I am not talking about zuul v3 or zuul v2.5 here | 14:18 |
| mmedvede | I think some of it is changing there | 14:18 |
| mmedvede | isaacb: you can check gearman queue status with 'echo status | nc <gearmanserver> 4730 | 14:19 |
| isaacb | mmedvede: yes. I know. for example i see the following: | 14:21 |
| isaacb | build:xiv-tempest-dsvm-full-iscsi 3 3 9 | 14:21 |
| isaacb | mmedvede: but what happen if there are more jobs than free workers? | 14:21 |
| mmedvede | isaacb: then, for your example, you would see: | 14:22 |
| mmedvede | build:xiv-tempest-dsvm-full-iscsi 20 9 9 | 14:22 |
| mmedvede | which would mean you have 9 workers, all 9 are being used, and you have 11 queued and waiting for a worker | 14:23 |
| isaacb | mmedvede: ok. | 14:23 |
| mmedvede | isaacb: what sorts of failures you see? | 14:24 |
| mmedvede | are they due to VM being reused again? e.g. with dirty state? | 14:25 |
| isaacb | mmedvede: from jenkins point of view, is there a chance that he tries to run a job on a slave which he already use rather than a slave that is idle? | 14:25 |
| mmedvede | isaacb: yes, it is possible | 14:25 |
| mmedvede | isaacb: for example, if it does not delete a VM on completion | 14:26 |
| mmedvede | isaacb: also, if you manually rebuild from jenkins, it could cause problems | 14:26 |
| isaacb | mmedvede: I see that jenkins is trying to use the same slave twice. during normal execution | 14:27 |
| isaacb | mmedvede: (no manual rebuild) | 14:28 |
| *** krtaylor has quit IRC | 14:28 | |
| isaacb | mmedvede: this happens only if I get to the maximum capacity of nodepool | 14:28 |
| *** krtaylor has joined #openstack-third-party-ci | 14:29 | |
| mmedvede | isaacb: do you set OFFLINE_NODE_WHEN_COMPLETE=1 ? | 14:30 |
| mmedvede | isaacb: you can check by going to any build, "Parameters" | 14:30 |
| isaacb | I can't see this parameter. | 14:32 |
| isaacb | mmedvede: who sets this parameter? | 14:33 |
| mmedvede | zuul | 14:33 |
| mmedvede | isaacb: do you see any parameters there? | 14:34 |
| mmedvede | e.g. ZUUL_PROJECT? | 14:34 |
| isaacb | yes | 14:34 |
| isaacb | mmedvede: I see many ZUUL's parameters but not this one | 14:35 |
| mmedvede | isaacb: https://github.com/openstack-infra/project-config-example/blob/master/zuul/openstack_functions.py#L34 | 14:35 |
| mmedvede | and the py file hooked up here https://github.com/openstack-infra/project-config-example/blob/master/zuul/layout.yaml#L3 | 14:36 |
| isaacb | mmedvede: ok | 14:36 |
| mmedvede | isaacb: and also need to set the function to use https://github.com/openstack-infra/project-config-example/blob/master/zuul/layout.yaml#L3 | 14:38 |
| isaacb | :q | 15:09 |
| isaacb | mmedvede: are you sure I should see OFFLINE_NODE_WHEN_COMPLETE in the parameters list in jenkins? | 15:11 |
| mmedvede | yes. If it is not there, it is not being set | 15:11 |
| *** asselin has joined #openstack-third-party-ci | 15:17 | |
| *** asselin_ has quit IRC | 15:20 | |
| isaacb | mmedvede: I didn't have the jobs section in the layout file. that's why the function in openstack_function.py wasn't called | 15:33 |
| isaacb | mmedvede: hope now I'll get this parameter | 15:34 |
| mmedvede | good, let me know if it solves the original problem | 15:35 |
| *** isaacb_ has joined #openstack-third-party-ci | 16:21 | |
| *** isaacb_ has quit IRC | 16:22 | |
| *** isaacb has quit IRC | 16:25 | |
| *** devlaps has joined #openstack-third-party-ci | 16:43 | |
| *** Apoorva has joined #openstack-third-party-ci | 17:38 | |
| *** rajinir has joined #openstack-third-party-ci | 17:39 | |
| *** strigazi is now known as strigazi_AFK | 17:46 | |
| *** krtaylor has quit IRC | 18:06 | |
| *** krtaylor has joined #openstack-third-party-ci | 19:17 | |
| *** rfolco has quit IRC | 20:18 | |
| *** krtaylor has quit IRC | 21:58 | |
| *** krtaylor has joined #openstack-third-party-ci | 22:42 | |
| *** rfolco has joined #openstack-third-party-ci | 23:15 | |
| *** rfolco has quit IRC | 23:15 | |
| *** viks_ has joined #openstack-third-party-ci | 23:45 | |
| *** stakeda has quit IRC | 23:48 | |
| *** stakeda has joined #openstack-third-party-ci | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!