| *** jamesmcarthur has joined #openstack-meeting-3 | 00:30 | |
| *** jamesmcarthur has quit IRC | 00:50 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 00:51 | |
| *** jamesmcarthur has quit IRC | 00:58 | |
| *** diablo_rojo has quit IRC | 01:01 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 01:14 | |
| *** jamesmcarthur has quit IRC | 01:23 | |
| *** markvoelker has joined #openstack-meeting-3 | 01:30 | |
| *** markvoelker has quit IRC | 01:35 | |
| *** yamamoto has joined #openstack-meeting-3 | 01:40 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 01:42 | |
| *** jamesmcarthur has quit IRC | 01:50 | |
| *** yamamoto has quit IRC | 02:04 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 02:11 | |
| *** jamesmcarthur has quit IRC | 02:11 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 02:11 | |
| *** yamamoto has joined #openstack-meeting-3 | 02:37 | |
| *** jamesmcarthur has quit IRC | 02:57 | |
| *** huyupeng has joined #openstack-meeting-3 | 03:01 | |
| huyupeng | will large scale meeting be held at CST 16:00 today ? | 03:02 |
|---|---|---|
| *** jamesmcarthur has joined #openstack-meeting-3 | 03:02 | |
| *** artom has joined #openstack-meeting-3 | 03:07 | |
| *** markvoelker has joined #openstack-meeting-3 | 03:31 | |
| *** markvoelker has quit IRC | 03:36 | |
| *** huyupeng has quit IRC | 03:36 | |
| *** psachin has joined #openstack-meeting-3 | 03:43 | |
| *** jamesmcarthur has quit IRC | 04:31 | |
| *** markvoelker has joined #openstack-meeting-3 | 04:34 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 04:37 | |
| *** markvoelker has quit IRC | 04:39 | |
| *** apetrich has quit IRC | 04:43 | |
| *** jamesmcarthur has quit IRC | 04:58 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 04:58 | |
| *** jamesmcarthur has quit IRC | 05:04 | |
| *** links has joined #openstack-meeting-3 | 05:19 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 05:33 | |
| *** jamesmcarthur has quit IRC | 05:42 | |
| *** huyupeng has joined #openstack-meeting-3 | 06:09 | |
| *** slaweq_ has joined #openstack-meeting-3 | 06:12 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 06:13 | |
| *** slaweq has quit IRC | 06:13 | |
| *** huyupeng has quit IRC | 06:14 | |
| *** markvoelker has joined #openstack-meeting-3 | 06:14 | |
| *** markvoelker has quit IRC | 06:20 | |
| *** jamesmcarthur has quit IRC | 06:22 | |
| *** ralonsoh has joined #openstack-meeting-3 | 06:55 | |
| *** huyupeng has joined #openstack-meeting-3 | 07:21 | |
| *** maciejjozefczyk has joined #openstack-meeting-3 | 07:29 | |
| *** belmoreira has joined #openstack-meeting-3 | 07:49 | |
| *** slaweq_ is now known as slaweq | 07:49 | |
| ttx | huyupeng: we'll meet starting in 10 minutes yes | 07:50 |
| *** brinzhang_ has joined #openstack-meeting-3 | 07:53 | |
| *** weichch has joined #openstack-meeting-3 | 07:57 | |
| ttx | #startmeeting large_scale_sig | 08:00 |
| openstack | Meeting started Wed Jun 24 08:00:02 2020 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:00 |
| *** openstack changes topic to " (Meeting topic: large_scale_sig)" | 08:00 | |
| ttx | #topic Rollcall | 08:00 |
| openstack | The meeting name has been set to 'large_scale_sig' | 08:00 |
| *** openstack changes topic to "Rollcall (Meeting topic: large_scale_sig)" | 08:00 | |
| ttx | Who is here for the Large Scale SIG meeting ? | 08:00 |
| amorin | hello! | 08:00 |
| ttx | belmoreira: ping | 08:00 |
| belmoreira | o/ | 08:00 |
| belmoreira | ttx: thanks for the ping | 08:01 |
| ttx | huyupeng: hi! | 08:01 |
| huyupeng | hello | 08:01 |
| ttx | huyupeng: first time joining? | 08:01 |
| huyupeng | yes | 08:01 |
| brinzhang_ | \o | 08:02 |
| ttx | huyupeng: care to introduce yourself, and let us know what you're interested in? | 08:02 |
| ttx | Or we can jump right in today's meeting agenda: | 08:03 |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-meeting | 08:03 |
| huyupeng | ok, i am intersted in concurrency, resource utilize , data plane performance when we deploy large scale openstack cloud | 08:04 |
| ttx | huyupeng: sounds like you found the right group to discuss it. I think those are concerns that the group shares | 08:04 |
| ttx | #topic 'Large scale operations' OpenDev event next week | 08:04 |
| *** openstack changes topic to "'Large scale operations' OpenDev event next week (Meeting topic: large_scale_sig)" | 08:05 | |
| ttx | #link https://www.openstack.org/events/opendev-2020/ | 08:05 |
| ttx | Are y'all ready for the event next week? | 08:05 |
| ttx | Remember that our goal is to start new discussions, and motivate new organizations to engage publicly | 08:05 |
| ttx | The Large Scale SIG should be used to continue those discussions, we don't want this to be a one-off thing without follow-up | 08:05 |
| ttx | So the idea will be to heavily promote the SIG during all those discussions | 08:06 |
| ttx | We should point to our wiki page | 08:06 |
| *** pandaboy has joined #openstack-meeting-3 | 08:06 | |
| ttx | #link https://wiki.openstack.org/wiki/Large_Scale_SIG | 08:06 |
| ttx | We should also point to our next meeting, July 8, 8utc on #openstack-meeting-3 | 08:06 |
| ttx | And also to the meeting etherpad for people to add items to the agenda. | 08:06 |
| ttx | belmoreira: Anything else we need to prepare before next week event? | 08:06 |
| belmoreira | we have the "Upgrades" session on Monday. It would be great to have all your experience there for discussion | 08:07 |
| *** belmoreira has quit IRC | 08:08 | |
| *** belmoreira has joined #openstack-meeting-3 | 08:09 | |
| ttx | Also don;t forget to register, it's free | 08:09 |
| huyupeng | i care about how rabbitmq behavie when there are a lot client publish message and receive message. how to tunning the rabbitmq | 08:09 |
| belmoreira | sorry got disconnected... not sure if you got my msg | 08:09 |
| belmoreira | we have the "Upgrades" session on Monday. It would be great to have all your experience there for discussion | 08:09 |
| amorin | belmoreira: ack | 08:09 |
| amorin | huyupeng: we all struggle on rabbitmq tuning :p | 08:10 |
| ttx | huyupeng: we identified this as a scaling bottleneck too. We are working to instrument oslo.messaging to get better data | 08:10 |
| ttx | in a new library called oslo.metrics | 08:10 |
| ttx | we'll talk about it in a minute | 08:10 |
| ttx | #topic Progress on "Documenting large scale operations" goal | 08:10 |
| *** openstack changes topic to "Progress on "Documenting large scale operations" goal (Meeting topic: large_scale_sig)" | 08:10 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-documentation | 08:10 |
| ttx | For the newcomers, this is one of our current goals for the SIG - produce better documentation to help operators setting up large deployments. | 08:10 |
| ttx | In particular, documenting better configuration values when you start hitting issues with default values at a certain scale | 08:11 |
| ttx | amorin has been leading that effort | 08:11 |
| amorin | I havn't be able to move forward on that topic in the past 2 weeks | 08:11 |
| ttx | OK I'll push the action item back | 08:11 |
| amorin | so if we can copy back my action, | 08:11 |
| ttx | #action amorin to add some meat to the wiki page before we push the Nova doc patch further | 08:11 |
| amorin | thanks | 08:11 |
| ttx | We also had a TODO for all to describe briefly how you solved metrics/billing in your deployment | 08:11 |
| ttx | But not much contribution to that yet | 08:11 |
| amorin | completely forgot about that | 08:12 |
| ttx | huyupeng: one objective of the SIG is to share experience | 08:12 |
| amorin | do we have a etherpad to fill? | 08:12 |
| ttx | like how various groups solved common problems | 08:12 |
| ttx | I added it into https://etherpad.openstack.org/p/large-scale-sig-documentation | 08:12 |
| amorin | ack | 08:12 |
| ttx | feel free to drop a line there | 08:12 |
| ttx | I'll keep that work item open too | 08:13 |
| ttx | #action all to describe briefly how you solved metrics/billing in your deployment in https://etherpad.openstack.org/p/large-scale-sig-documentation | 08:13 |
| ttx | There seems to be a lot of custom solution in that space, which points to shortcomings in the openstack services offering | 08:13 |
| ttx | Anything else on this topic? | 08:14 |
| ttx | #topic Progress on "Scaling within one cluster" goal | 08:15 |
| *** openstack changes topic to "Progress on "Scaling within one cluster" goal (Meeting topic: large_scale_sig)" | 08:15 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling | 08:15 |
| ttx | For the newcomers, this is the other current goal of the SIG - identify, measure and push back common scaling limitations within one cluster | 08:15 |
| ttx | Like RabbitMQ starting to fall off first | 08:15 |
| *** markvoelker has joined #openstack-meeting-3 | 08:16 | |
| ttx | One key effort within that goal is to work on bringing LINE's work on instrumenting oslo.messaging into upstream OpenStack | 08:16 |
| ttx | I had a TODO on solving merge conflict and push initial oslo.metrics code in | 08:16 |
| ttx | It's now in at: https://opendev.org/openstack/oslo.metrics/ | 08:16 |
| ttx | It's mostly at POC stage, but would be great if this group could look it up and tell us if that sounds like a good idea, and how to improve it | 08:17 |
| ttx | We also had a bunch of patches in to make the code more OpenStacky, thanks to hberaud | 08:17 |
| ttx | smcginnis and I did the reviews. | 08:17 |
| ttx | I think for the more feature-oriented reviews coming in the future, it would be good to have one of the original authors from LINE in the core review team for oslo-metrics | 08:17 |
| ttx | I'll try to see with masahito if he would be interested in being added | 08:17 |
| ttx | The next step will be to add basic tests so that we are reasonably confident we do not introduce regressions | 08:18 |
| ttx | I suspect masahito will mention it next week during the Opendev event | 08:18 |
| *** jamesmcarthur has joined #openstack-meeting-3 | 08:18 | |
| ttx | Anything else on that topic? | 08:19 |
| huyupeng | does oslo.metrics has something to do with rabbitmq tunning ? | 08:19 |
| ttx | yes, let me point to teh presentation LINE did at the Summit in Shanghai | 08:20 |
| *** markvoelker has quit IRC | 08:21 | |
| ttx | https://www.openstack.org/videos/shanghai-2019/how-we-used-rabbitmq-in-wrong-way-at-a-scale | 08:21 |
| ttx | That will give you the background | 08:21 |
| ttx | #topic Large Scale SIG slide for OpenStack 10th birthday campaign | 08:21 |
| *** openstack changes topic to "Large Scale SIG slide for OpenStack 10th birthday campaign (Meeting topic: large_scale_sig)" | 08:21 | |
| ttx | Helena on the OSF staff reached out to SIG with the following information: | 08:21 |
| huyupeng | thanks, i will see later | 08:21 |
| ttx | "OpenStack will be turning 10 years old in less than a month! We are sending out a slide deck to the event organizers who are organizing the 10 years of OpenStack celebration. For this slide deck, is there anything about the Large Scale SIG you would like to include?" | 08:21 |
| ttx | I was thinking we could make a slide with high-level goals for the SIG and pointers to our meeting time | 08:22 |
| ttx | Anyone interested in working on that? Or I can make a draft and propose it next meeting | 08:22 |
| ttx | #action ttx to produce a draft of the 10th birthday slide for the SIG | 08:22 |
| *** jamesmcarthur has quit IRC | 08:23 | |
| ttx | huyupeng: in the etherpad links I provided above, you will find other ways to get involved in those SIG goals, like providing scaling stories | 08:23 |
| ttx | Scaling stories are accounts of scaling experiences in your deployment -- what broke first, how did it break, and how you moved past it | 08:24 |
| huyupeng | ok,copy that | 08:24 |
| ttx | We hope to find common pain points this way (RabbitMQ being an obvious one) | 08:24 |
| ttx | We document those in https://etherpad.openstack.org/p/scaling-stories | 08:25 |
| ttx | and then copy them to https://wiki.openstack.org/wiki/Large_Scale_Scaling_Stories for long-term storage | 08:25 |
| ttx | We hope to have more starting next week after the Opendev event! | 08:25 |
| ttx | #topic Next meeting | 08:26 |
| *** openstack changes topic to "Next meeting (Meeting topic: large_scale_sig)" | 08:26 | |
| *** pandaboy has quit IRC | 08:26 | |
| ttx | Next meeting in two weeks... That's the week just after the Opendev event, so if we get contacts of people excited to join the SIG, we can strike that iron while it's hot. | 08:26 |
| ttx | #info next meeting: Jul 8, 8:00UTC | 08:26 |
| amorin | ok | 08:26 |
| ttx | Anything else before we close ? | 08:26 |
| ttx | If there is a topic you want to discuss next time, make sure to add it in advance to the meeting agenda on https://etherpad.opendev.org/p/large-scale-sig-meeting ! | 08:27 |
| ttx | Alright, if nothing else for today, have a great day, and hope to see you all virtually at Opendev next week | 08:28 |
| ttx | Thanks everyone! | 08:28 |
| ttx | #endmeeting | 08:28 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 08:28 | |
| openstack | Meeting ended Wed Jun 24 08:28:55 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:28 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-06-24-08.00.html | 08:28 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-06-24-08.00.txt | 08:28 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-06-24-08.00.log.html | 08:29 |
| amorin | thanks, bye | 08:29 |
| *** mdelavergne has joined #openstack-meeting-3 | 08:33 | |
| *** weichch has quit IRC | 08:42 | |
| *** apetrich has joined #openstack-meeting-3 | 08:43 | |
| *** huyupeng has quit IRC | 08:45 | |
| *** e0ne has joined #openstack-meeting-3 | 09:23 | |
| *** brinzhang_ has quit IRC | 09:52 | |
| *** markvoelker has joined #openstack-meeting-3 | 10:05 | |
| *** markvoelker has quit IRC | 10:09 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 10:20 | |
| *** jamesmcarthur has quit IRC | 10:32 | |
| *** psachin has quit IRC | 10:57 | |
| *** raildo has joined #openstack-meeting-3 | 11:21 | |
| *** artom has quit IRC | 11:35 | |
| *** mdelavergne has quit IRC | 11:49 | |
| *** markvoelker has joined #openstack-meeting-3 | 11:55 | |
| *** markvoelker has quit IRC | 11:59 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 12:29 | |
| *** jamesmcarthur has quit IRC | 12:34 | |
| *** liuyulong has joined #openstack-meeting-3 | 14:06 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 14:15 | |
| *** links has quit IRC | 14:38 | |
| *** jamesmcarthur has quit IRC | 14:54 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 14:54 | |
| slaweq | #startmeeting neutron_ci | 15:00 |
| openstack | Meeting started Wed Jun 24 15:00:27 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:00 | |
| slaweq | hi | 15:00 |
| openstack | The meeting name has been set to 'neutron_ci' | 15:00 |
| ralonsoh | hi | 15:00 |
| bcafarel | hello | 15:01 |
| slaweq | first of all | 15:01 |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:01 |
| *** lajoskatona has joined #openstack-meeting-3 | 15:01 | |
| slaweq | please open now to have it ready for later :) | 15:02 |
| lajoskatona | o/ | 15:02 |
| slaweq | ok, lets start | 15:03 |
| maciejjozefczyk | \o | 15:03 |
| slaweq | #topic Actions from previous meetings | 15:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:03 | |
| slaweq | we have only one action from last week | 15:03 |
| slaweq | slaweq to add additional logging for fullstack's firewall tests | 15:03 |
| slaweq | I found the issue again in https://zuul.opendev.org/t/openstack/build/c5451e9e66fe4c14b2a09339a77fc449 | 15:04 |
| slaweq | After checking it seems for me that failure was at the beginning of the test, when connectivity with ncat was checked | 15:04 |
| slaweq | I proposed patch https://review.opendev.org/#/c/737741/ for now. | 15:04 |
| slaweq | After that lets see what will happen more | 15:04 |
| njohnston | o/ | 15:04 |
| slaweq | that's all update from me about this | 15:05 |
| slaweq | #topic Stadium projects | 15:05 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 15:05 | |
| slaweq | I don't have any updates about stadium projects today | 15:05 |
| slaweq | but maybe You have something | 15:05 |
| lajoskatona | nothing special from me, but if you have time please look there for things to review :-) | 15:06 |
| bcafarel | not from me, we had enough CI trouble in neutron itself to keep busy | 15:06 |
| njohnston | nothing for me either | 15:06 |
| slaweq | ok, sure lajoskatona I will check list of opened patches in stadium projects | 15:06 |
| slaweq | ok, so next topic | 15:06 |
| slaweq | #topic Stable branches | 15:07 |
| *** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)" | 15:07 | |
| slaweq | Ussuri dashboard: http://grafana.openstack.org/d/pM54U-Kiz/neutron-failure-rate-previous-stable-release?orgId=1 | 15:07 |
| slaweq | Train dashboard: http://grafana.openstack.org/d/dCFVU-Kik/neutron-failure-rate-older-stable-release?orgId=1 | 15:07 |
| slaweq | for ussuri and train it seems that it works fine now | 15:07 |
| lajoskatona | slaweq: thanks | 15:07 |
| slaweq | but for older releases, like rocky and queens, I think we still have uwsgi issue | 15:07 |
| slaweq | bcafarel: are You aware of it? | 15:07 |
| slaweq | I got many failures e.g. on https://review.opendev.org/#/c/737703/ today | 15:07 |
| bcafarel | yes for EM branches we have lingering issue with grenade jobs ( gmann mentioned them in recent mail update) | 15:08 |
| bcafarel | https://review.opendev.org/737414 should workaround it for the time being | 15:08 |
| bcafarel | once it works, and backported to older branches | 15:08 |
| slaweq | bcafarel: but it's only for grenade | 15:10 |
| slaweq | and in my patch I saw failures in all tempest jobs too | 15:10 |
| slaweq | basicaly everything except UT and pep8 was red | 15:10 |
| bcafarel | oh sigh | 15:10 |
| bcafarel | I did not check rocky thoroughly yet, still on stein :/ | 15:11 |
| slaweq | I see error like: "ls: cannot access 'uwsgi*': No such file or directory" in devstack log | 15:11 |
| *** jamesmcarthur has quit IRC | 15:12 | |
| bcafarel | ack there was mention on uwsgi + rocky and older still WIP http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015558.html | 15:13 |
| slaweq | bcafarel: do You have cycles to check that this week? | 15:13 |
| *** jamesmcarthur has joined #openstack-meeting-3 | 15:13 | |
| bcafarel | slaweq: yes, at least to get some up-to-date status on it! | 15:13 |
| slaweq | thx | 15:13 |
| slaweq | #action bcafarel to check gate status on rocky and queens (uwsgi problem) | 15:13 |
| slaweq | ok | 15:14 |
| slaweq | lets move on | 15:14 |
| slaweq | #topic Grafana | 15:14 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:14 | |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:14 |
| slaweq | in master branch it looks ok this week IMO | 15:15 |
| njohnston | +1 | 15:17 |
| slaweq | so lets talk about some issues in specific jobs | 15:18 |
| slaweq | #topic fullstack/functional | 15:18 |
| *** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)" | 15:18 | |
| slaweq | first functional tests | 15:18 |
| slaweq | I found today some db migration errors (again), like https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ec3/736269/3/check/neutron-functional-with-uwsgi/ec30046/testr_results.html | 15:18 |
| slaweq | ralonsoh: is it something what You want to address with Your db migration script changes? | 15:18 |
| ralonsoh | yes | 15:19 |
| ralonsoh | those errors in test_walk_versions | 15:19 |
| ralonsoh | and test_has_offline_migrations_all_heads_upgraded | 15:19 |
| slaweq | ok | 15:19 |
| slaweq | in general I think that those failures are due to slow node where it was run | 15:20 |
| slaweq | but will be good if we can get rid of at least some of them | 15:20 |
| slaweq | I also have other issue | 15:21 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.html | 15:21 |
| slaweq | this seems for me like somethig related to our privileged ip_lib or pyroute2 | 15:21 |
| slaweq | as there are issues that interfaces are not found in namespace None | 15:22 |
| slaweq | and there are also issue with _check_bridge_datapath_id() method in tests there | 15:22 |
| ralonsoh | well, maybe that's correct and the interface was not created | 15:22 |
| *** jamesmcarthur has quit IRC | 15:23 | |
| slaweq | ralonsoh: can be | 15:23 |
| slaweq | so at least we should probably fix exception message IMO | 15:24 |
| maciejjozefczyk | there is also this failure test_ovsdb_monitor.TestNBDbMonitorOverTcp.test_floatingip_mac_bindings (IndexError: list index out of range)... I can take a look on that one | 15:24 |
| slaweq | or not, nvm :) | 15:24 |
| slaweq | thx maciejjozefczyk | 15:24 |
| ralonsoh | I can take a look at the bridge.get_datapath_id problem | 15:25 |
| ralonsoh | if I have some time this week | 15:25 |
| slaweq | thx ralonsoh | 15:25 |
| slaweq | #action maciejjozefczyk will check test_ovsdb_monitor.TestNBDbMonitorOverTcp.test_floatingip_mac_bindings failiure in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.html | 15:25 |
| slaweq | #action ralonsoh will check get_datapath_id issues in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.htm | 15:26 |
| slaweq | #undo | 15:26 |
| openstack | Removing item from minutes: #action ralonsoh will check get_datapath_id issues in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.htm | 15:26 |
| slaweq | #action ralonsoh will check get_datapath_id issues in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.html | 15:26 |
| slaweq | I will check this errors with non existing devices | 15:27 |
| slaweq | #action slaweq will check errors with non existing interface in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_63a/711425/11/check/neutron-functional/63ac4ca/testr_results.html | 15:28 |
| slaweq | ok, lets move on | 15:28 |
| slaweq | #topic Tempest/Scenario | 15:28 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 15:28 | |
| slaweq | in scenario tests I found one interesting issue this week | 15:29 |
| slaweq | it happend only once I think but maybe worth to deeper look | 15:29 |
| slaweq | https://11b28b714aaa0f2eaa01-115c1089095738e3e088969e8724f0ca.ssl.cf1.rackcdn.com/712640/9/check/neutron-tempest-plugin-scenario-openvswitch/2ad316a/testr_results.html | 15:29 |
| slaweq | error with address already allocated in subnet | 15:29 |
| slaweq | maybe some bug in tests? | 15:29 |
| ralonsoh | didn't I sent a patch for this? | 15:30 |
| ralonsoh | send* | 15:30 |
| slaweq | ralonsoh: I don't remember | 15:30 |
| ralonsoh | (checking) | 15:30 |
| slaweq | ahh, right | 15:30 |
| slaweq | I think that now I remember | 15:30 |
| ralonsoh | https://review.opendev.org/#/c/731267/ | 15:30 |
| slaweq | ok, this failure was before Your patch was merged | 15:31 |
| slaweq | so we should be good now | 15:31 |
| slaweq | thx ralonsoh | 15:31 |
| ralonsoh | ahhh ok perfect | 15:31 |
| bcafarel | ralonsoh: fixing CI issues before slaweq complains about them, nice! | 15:32 |
| ralonsoh | hahaha | 15:32 |
| slaweq | haha | 15:32 |
| slaweq | true | 15:32 |
| maciejjozefczyk | ralonsoh++ :D | 15:32 |
| slaweq | ok, so one last topic for today | 15:32 |
| slaweq | #topic Periodic | 15:32 |
| *** openstack changes topic to "Periodic (Meeting topic: neutron_ci)" | 15:32 | |
| slaweq | all except one job looks fine there | 15:33 |
| slaweq | but neutron-ovn-tempest-ovs-master-fedora is broken again | 15:33 |
| maciejjozefczyk | sigh | 15:33 |
| slaweq | and it's broken on compilation of ovs | 15:33 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b03/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-ovs-master-fedora/b039158/job-output.txt | 15:33 |
| *** yamamoto has quit IRC | 15:34 | |
| slaweq | maciejjozefczyk: can You take a look into that one? | 15:34 |
| maciejjozefczyk | slaweq, yes | 15:35 |
| slaweq | thx | 15:35 |
| maciejjozefczyk | slaweq, I fixed that failure some time ago and it is failing again.. strange | 15:35 |
| maciejjozefczyk | I'll take a closer look on that, thanks | 15:35 |
| slaweq | #action maciejjozefczyk to check failing neutron-ovn-tempest-ovs-master-fedora periodic job | 15:35 |
| slaweq | thx a lot | 15:35 |
| slaweq | ok, that's all from me today | 15:35 |
| slaweq | (that was fast ;)) | 15:35 |
| slaweq | anything else You want to discuss today? | 15:36 |
| ralonsoh | no thanks | 15:36 |
| slaweq | if not, I think I can give You back about 20 minutes today :) | 15:37 |
| bcafarel | I just realized "stadium projects" section we had also community goals mixed in | 15:37 |
| bcafarel | short update on that I started to fill https://etherpad.opendev.org/p/neutron-victoria-switch_to_focal | 15:37 |
| bcafarel | hopefully it will be more complete by next week :) | 15:38 |
| slaweq | thx bcafarel | 15:38 |
| lajoskatona | bcafarel: good point, | 15:38 |
| slaweq | btw, speaking about focal, it seems from https://review.opendev.org/#/c/737370/ that neutron is running fine there | 15:38 |
| lajoskatona | I checked with odl but I got weird failures, and thought to wait some time till the waves go down :-) | 15:38 |
| slaweq | there are some problems realated to cinder in tempest jobs | 15:39 |
| lajoskatona | bcafarel, slaweq: for odl I had this: https://review.opendev.org/736703 | 15:40 |
| bcafarel | lajoskatona: indeed, main patches in devstack and tempest should clear out most of issues - and most of our jobs will inherit directly | 15:40 |
| lajoskatona | and it fails with nodeset not found or similar | 15:40 |
| lajoskatona | bcafarel: that was my feeling as well :-) | 15:41 |
| bcafarel | lajoskatona: or depending on https://review.opendev.org/734700 should help to get the nodes and job definitions | 15:41 |
| slaweq | thx bcafarel and lajoskatona | 15:41 |
| lajoskatona | bcafarel: thanks, I do a try with that, the wf -1 though a little frightening | 15:42 |
| lajoskatona | but that's perhaps just some timing from gmann | 15:42 |
| bcafarel | yes it is in "heavy progress" there | 15:44 |
| bcafarel | though just for inheriting the nodeset it should be safe enough for testing | 15:44 |
| gmann | yeah, 734700 is right to test. i was waiting for gate result there and i will announce the same on ML also. | 15:44 |
| lajoskatona | gmann, bcafarel: thanks, I check it | 15:44 |
| slaweq | gmann: in our tests in https://review.opendev.org/#/c/737370/ I saw some cinder related failures | 15:45 |
| slaweq | gmann: do You know if cinder team is aware of them? | 15:45 |
| gmann | bcafarel: if your job override nodeset then you need to change otherwise 734700 can take care of devstack base job nodeset switch to focal | 15:45 |
| slaweq | or maybe I should open LP for them? | 15:45 |
| gmann | slaweq: current known bug is this https://bugs.launchpad.net/nova/+bug/1882521 | 15:45 |
| openstack | Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [Undecided,New] | 15:45 |
| bcafarel | gmann: ack I am making a list of jobs that will need some action (more than just 734700) | 15:46 |
| gmann | which is volume detach issue but i have not checked your patch failure | 15:46 |
| gmann | bcafarel: +1 | 15:46 |
| slaweq | gmann: seems like the same one | 15:47 |
| slaweq | thx | 15:47 |
| slaweq | ok, I think we can finish meeting now | 15:49 |
| slaweq | thx for attending | 15:49 |
| maciejjozefczyk | \o | 15:49 |
| slaweq | and see You o/ | 15:49 |
| slaweq | #endmeeting | 15:49 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:50 | |
| openstack | Meeting ended Wed Jun 24 15:49:59 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:50 |
| lajoskatona | Bye | 15:50 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-06-24-15.00.html | 15:50 |
| ralonsoh | byw | 15:50 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-06-24-15.00.txt | 15:50 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-06-24-15.00.log.html | 15:50 |
| *** lajoskatona has left #openstack-meeting-3 | 15:50 | |
| bcafarel | o/ | 15:50 |
| *** ricolin has joined #openstack-meeting-3 | 15:55 | |
| *** liuyulong has quit IRC | 15:56 | |
| *** yamamoto has joined #openstack-meeting-3 | 16:04 | |
| *** yamamoto has quit IRC | 16:10 | |
| *** markvoelker has joined #openstack-meeting-3 | 16:24 | |
| *** markvoelker has quit IRC | 16:28 | |
| *** markvoelker has joined #openstack-meeting-3 | 17:09 | |
| *** markvoelker has quit IRC | 17:14 | |
| *** ricolin has quit IRC | 17:20 | |
| *** apetrich has quit IRC | 17:20 | |
| *** apetrich has joined #openstack-meeting-3 | 17:24 | |
| *** e0ne has quit IRC | 17:42 | |
| *** apetrich has quit IRC | 18:15 | |
| *** apetrich has joined #openstack-meeting-3 | 18:16 | |
| *** diablo_rojo has joined #openstack-meeting-3 | 18:23 | |
| *** belmoreira has quit IRC | 18:33 | |
| *** irclogbot_0 has quit IRC | 18:34 | |
| *** irclogbot_3 has joined #openstack-meeting-3 | 18:38 | |
| *** yamamoto has joined #openstack-meeting-3 | 18:53 | |
| *** yamamoto has quit IRC | 18:58 | |
| *** markvoelker has joined #openstack-meeting-3 | 19:10 | |
| *** markvoelker has quit IRC | 19:15 | |
| *** ralonsoh has quit IRC | 19:26 | |
| *** markvoelker has joined #openstack-meeting-3 | 19:33 | |
| *** markvoelker has quit IRC | 19:38 | |
| *** markvoelker has joined #openstack-meeting-3 | 19:59 | |
| *** markvoelker has quit IRC | 20:04 | |
| *** belmoreira has joined #openstack-meeting-3 | 20:21 | |
| *** belmoreira has quit IRC | 20:26 | |
| *** maciejjozefczyk has quit IRC | 21:08 | |
| *** raildo has quit IRC | 21:09 | |
| *** slaweq has quit IRC | 21:36 | |
| *** slaweq has joined #openstack-meeting-3 | 21:46 | |
| *** slaweq has quit IRC | 21:52 | |
| *** markvoelker has joined #openstack-meeting-3 | 22:00 | |
| *** markvoelker has quit IRC | 22:05 | |
| *** markvoelker has joined #openstack-meeting-3 | 22:49 | |
| *** markvoelker has quit IRC | 22:54 | |
| *** markvoelker has joined #openstack-meeting-3 | 23:37 | |
| *** markvoelker has quit IRC | 23:47 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!