Wednesday, 2019-12-11

*** epei has quit IRC00:00
*** ykatabam has quit IRC00:03
*** ykatabam has joined #openstack-meeting00:03
*** rf0lc0 has quit IRC00:10
*** igordc has quit IRC00:13
*** jamesmcarthur has joined #openstack-meeting00:21
*** epei has joined #openstack-meeting00:27
*** epei has quit IRC00:32
*** jamesmcarthur has quit IRC00:39
*** epei has joined #openstack-meeting00:42
*** epei has quit IRC00:46
*** epei has joined #openstack-meeting00:47
*** epei has quit IRC00:51
*** zhangchi has joined #openstack-meeting01:07
*** epei has joined #openstack-meeting01:07
zhangchi#startmeeting  tricircle01:07
openstackMeeting started Wed Dec 11 01:07:37 2019 UTC and is due to finish in 60 minutes.  The chair is zhangchi. Information about MeetBot at http://wiki.debian.org/MeetBot.01:07
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.01:07
*** openstack changes topic to " (Meeting topic: tricircle)"01:07
openstackThe meeting name has been set to 'tricircle'01:07
zhangchiMorning all, welcome back01:08
zhangchiTricircle bi-weekly meeting01:08
*** gyee has quit IRC01:09
zhangchiNeutron master has updated and tricircle devstack installation can not pass  recently01:10
*** epei has quit IRC01:11
*** Liang__ has joined #openstack-meeting01:12
*** epei has joined #openstack-meeting01:16
zhangchi#link https://bugs.launchpad.net/tricircle/+bug/185445001:18
openstackLaunchpad bug 1854450 in Tricircle "tricirlce-devstack-has-no-constants-module" [Undecided,New]01:18
*** epei has quit IRC01:21
*** epei has joined #openstack-meeting01:22
*** epei has quit IRC01:26
zhangchiThis bugs needs to be solved first01:26
zhangchi#issues https://etherpad.openstack.org/p/tricircle-ussuri-plan01:31
*** mmethot has quit IRC01:36
zhangchiMorning  epei01:42
zhangchiHere you are, how things going with your l3 network and container function01:42
*** epei has joined #openstack-meeting01:42
zhangchiHi epei01:46
*** mmethot has joined #openstack-meeting01:47
zhangchiIs the work l3 network and container function still be moved on01:47
*** epei has quit IRC01:47
zhangchiThe network seems  not good No response today a little sad01:48
zhangchiBye everyone01:49
zhangchi#endmeeting01:49
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"01:49
openstackMeeting ended Wed Dec 11 01:49:09 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)01:49
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-12-11-01.07.html01:49
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-12-11-01.07.txt01:49
openstackLog:            http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-12-11-01.07.log.html01:49
*** larainema has joined #openstack-meeting01:49
*** rcernin has quit IRC01:53
*** ykatabam has quit IRC01:53
*** zhangchi has quit IRC01:57
*** epei has joined #openstack-meeting01:57
*** epei has quit IRC02:01
*** epei has joined #openstack-meeting02:07
*** epei has quit IRC02:12
*** epei has joined #openstack-meeting02:12
*** epei has quit IRC02:16
*** epei has joined #openstack-meeting02:18
*** epei has quit IRC02:22
*** epei has joined #openstack-meeting02:32
*** epei has quit IRC02:37
*** epei has joined #openstack-meeting02:37
*** ricolin has joined #openstack-meeting02:40
*** epei has quit IRC02:42
*** ykatabam has joined #openstack-meeting02:54
*** rcernin has joined #openstack-meeting02:54
*** epei has joined #openstack-meeting02:57
*** epei has quit IRC03:01
*** epei has joined #openstack-meeting03:07
*** epei has quit IRC03:11
*** epei has joined #openstack-meeting03:17
*** epei has quit IRC03:21
*** epei has joined #openstack-meeting03:26
*** munimeha1 has quit IRC03:28
*** epei has quit IRC03:31
*** epei has joined #openstack-meeting03:36
*** psachin has joined #openstack-meeting03:40
*** epei has quit IRC03:41
*** epei has joined #openstack-meeting03:47
*** ijw_ has quit IRC03:49
*** epei has quit IRC03:51
*** epei has joined #openstack-meeting03:57
*** tetsuro has quit IRC03:58
*** tetsuro has joined #openstack-meeting03:59
*** epei has quit IRC04:01
*** epei has joined #openstack-meeting04:02
*** ociuhandu has joined #openstack-meeting04:04
*** epei has quit IRC04:07
*** ociuhandu has quit IRC04:08
*** epei has joined #openstack-meeting04:17
*** epei has quit IRC04:21
*** epei has joined #openstack-meeting04:22
*** epei has quit IRC04:26
*** diablo_rojo has quit IRC04:29
*** epei has joined #openstack-meeting04:37
*** epei has quit IRC04:41
*** epei has joined #openstack-meeting04:46
*** epei has quit IRC04:51
*** epei has joined #openstack-meeting04:56
*** epei has quit IRC05:01
*** dansmith has quit IRC05:10
*** epei has joined #openstack-meeting05:11
*** dansmith has joined #openstack-meeting05:14
*** epei has quit IRC05:16
*** epei has joined #openstack-meeting05:21
*** epei has quit IRC05:25
*** tetsuro has quit IRC05:28
*** epei has joined #openstack-meeting05:30
*** tetsuro has joined #openstack-meeting05:31
*** ianychoi has joined #openstack-meeting05:32
*** epei has quit IRC05:35
*** epei has joined #openstack-meeting05:36
*** epei has quit IRC05:40
*** epei has joined #openstack-meeting05:56
*** epei has quit IRC06:00
*** epei has joined #openstack-meeting06:01
*** tetsuro has quit IRC06:04
*** epei has quit IRC06:05
*** tetsuro has joined #openstack-meeting06:09
*** pcaruana has joined #openstack-meeting06:09
*** epei has joined #openstack-meeting06:11
*** epei has quit IRC06:17
*** epei has joined #openstack-meeting06:22
*** epei has quit IRC06:26
*** epei has joined #openstack-meeting06:32
*** epei has quit IRC06:37
*** epei has joined #openstack-meeting06:37
*** epei has quit IRC06:42
*** epei has joined #openstack-meeting06:52
*** epei has quit IRC06:57
*** epei has joined #openstack-meeting06:57
*** epei has quit IRC07:02
*** epei has joined #openstack-meeting07:07
*** ianychoi_ has joined #openstack-meeting07:08
*** ianychoi has quit IRC07:11
*** epei has quit IRC07:12
*** lpetrut has joined #openstack-meeting07:18
*** epei has joined #openstack-meeting07:22
*** epei has quit IRC07:26
*** epei has joined #openstack-meeting07:27
*** epei has quit IRC07:32
*** epei has joined #openstack-meeting07:37
*** epei has quit IRC07:42
*** gibi_off is now known as gibi07:44
*** slaweq has joined #openstack-meeting07:47
*** epei has joined #openstack-meeting07:52
*** belmoreira has joined #openstack-meeting07:56
*** epei has quit IRC07:56
*** tesseract has joined #openstack-meeting07:57
*** e0ne has joined #openstack-meeting07:59
*** epei has joined #openstack-meeting08:02
*** ociuhandu has joined #openstack-meeting08:05
*** ociuhandu has quit IRC08:05
*** e0ne has quit IRC08:06
*** epei has quit IRC08:07
*** ociuhandu has joined #openstack-meeting08:07
*** epei has joined #openstack-meeting08:07
*** epei has quit IRC08:12
*** epei has joined #openstack-meeting08:17
*** e0ne has joined #openstack-meeting08:19
*** epei has quit IRC08:21
*** pcaruana has quit IRC08:22
*** epei has joined #openstack-meeting08:22
*** epei has quit IRC08:26
*** epei has joined #openstack-meeting08:28
*** e0ne has quit IRC08:28
*** ociuhandu has quit IRC08:30
*** epei has quit IRC08:33
*** rpittau|afk is now known as rpittau08:36
*** epei has joined #openstack-meeting08:38
*** epei has quit IRC08:43
*** kopecmartin|off is now known as kopecmartin08:44
*** links has joined #openstack-meeting08:44
*** aloga has quit IRC08:45
*** aloga has joined #openstack-meeting08:45
*** epei has joined #openstack-meeting08:48
*** epei has quit IRC08:52
*** epei has joined #openstack-meeting08:53
*** ociuhandu has joined #openstack-meeting08:54
*** ralonsoh has joined #openstack-meeting08:55
*** strigazi has joined #openstack-meeting08:55
*** epei has quit IRC08:57
*** epei has joined #openstack-meeting08:58
*** ociuhandu has quit IRC08:59
*** ociuhandu has joined #openstack-meeting09:00
*** masahito has joined #openstack-meeting09:02
*** epei has quit IRC09:03
*** ociuhandu has quit IRC09:05
*** masahito has quit IRC09:06
*** epei has joined #openstack-meeting09:08
*** epei has quit IRC09:13
*** epei has joined #openstack-meeting09:18
*** masahito has joined #openstack-meeting09:21
*** epei has quit IRC09:22
*** epei has joined #openstack-meeting09:24
*** ykatabam has quit IRC09:26
*** ociuhandu has joined #openstack-meeting09:27
*** epei has quit IRC09:28
*** epei has joined #openstack-meeting09:29
*** epei has quit IRC09:33
*** epei has joined #openstack-meeting09:34
*** epei has quit IRC09:38
*** epei has joined #openstack-meeting09:39
*** epei has quit IRC09:43
*** epei has joined #openstack-meeting09:44
*** ttsiouts has joined #openstack-meeting09:47
*** epei has quit IRC09:49
*** masahito has quit IRC09:50
*** apetrich has joined #openstack-meeting09:50
*** rcernin has quit IRC09:50
*** farhanjamil has joined #openstack-meeting09:54
*** ianychoi_ has quit IRC09:56
*** epei has joined #openstack-meeting09:59
*** epei has quit IRC10:04
*** epei has joined #openstack-meeting10:05
*** e0ne has joined #openstack-meeting10:05
*** farhanjamil has quit IRC10:07
*** epei has quit IRC10:09
*** vishalmanchanda has quit IRC10:09
*** vishalmanchanda has joined #openstack-meeting10:10
*** Liang__ has quit IRC10:17
*** belmoreira has quit IRC10:44
*** belmoreira has joined #openstack-meeting10:45
*** belmoreira has quit IRC10:49
*** dmacpher__ has joined #openstack-meeting11:13
*** dmacpher_ has quit IRC11:16
*** ttsiouts has quit IRC11:24
*** raildo has joined #openstack-meeting11:32
*** pcaruana has joined #openstack-meeting11:36
*** ianychoi has joined #openstack-meeting11:49
*** ttsiouts has joined #openstack-meeting11:52
*** brinzhang has joined #openstack-meeting12:00
*** larainema has quit IRC12:01
*** brinzhang has quit IRC12:02
*** brinzhang has joined #openstack-meeting12:02
*** brinzhang has quit IRC12:03
*** brinzhang has joined #openstack-meeting12:03
*** brinzhang has quit IRC12:04
*** rfolco has joined #openstack-meeting12:05
*** brinzhang has joined #openstack-meeting12:07
*** brinzhang has quit IRC12:07
*** brinzhang has joined #openstack-meeting12:08
*** brinzhang has quit IRC12:09
*** brinzhang has joined #openstack-meeting12:11
*** brinzhang has quit IRC12:11
*** brinzhang has joined #openstack-meeting12:12
*** brinzhang_ has joined #openstack-meeting12:16
*** nicolasbock has joined #openstack-meeting12:18
*** brinzhang has quit IRC12:19
*** Liang__ has joined #openstack-meeting12:27
*** ociuhandu has quit IRC12:37
*** dmacpher__ has quit IRC12:47
*** dmacpher has joined #openstack-meeting12:47
*** ociuhandu has joined #openstack-meeting13:03
*** mmethot has quit IRC13:13
*** mmethot has joined #openstack-meeting13:17
*** anastzhyr has joined #openstack-meeting13:21
*** brinzhang_ has quit IRC13:33
*** Lucas_Gray has joined #openstack-meeting13:43
*** liuyulong has joined #openstack-meeting13:46
*** liuyulong_ has joined #openstack-meeting13:46
*** Lucas_Gray has quit IRC13:50
*** Lucas_Gray has joined #openstack-meeting13:51
*** Liang__ is now known as LiangFang13:54
liuyulong113:59
liuyulong_113:59
liuyulong#startmeeting neutron_l313:59
openstackMeeting started Wed Dec 11 13:59:32 2019 UTC and is due to finish in 60 minutes.  The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot.13:59
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:59
*** openstack changes topic to " (Meeting topic: neutron_l3)"13:59
openstackThe meeting name has been set to 'neutron_l3'13:59
liuyulong#chair liuyulong_13:59
openstackCurrent chairs: liuyulong liuyulong_13:59
*** mriedem has joined #openstack-meeting13:59
ralonsohhi14:00
slaweqhi14:00
liuyulonghi14:01
liuyulong#chair slaweq14:01
openstackCurrent chairs: liuyulong liuyulong_ slaweq14:01
*** bnemec has joined #openstack-meeting14:01
liuyulongOK, let's start14:01
liuyulong#topic Announcements14:01
*** openstack changes topic to "Announcements (Meeting topic: neutron_l3)"14:01
*** ociuhandu_ has joined #openstack-meeting14:02
liuyulongbad connection....14:04
liuyulong#link https://releases.openstack.org/ussuri/schedule.html14:04
liuyulongWe have this release schedule。14:04
liuyulongUssuri-1 is on Dec 09 - Dec 1314:04
*** ociuhandu has quit IRC14:05
*** kaisers has joined #openstack-meeting14:06
liuyulong#link https://launchpad.net/neutron/+milestone/ussuri-114:07
liuyulongOnly one bug was marked for U-1.14:07
slaweqliuyulong: this bug is actually RFE14:08
liuyulongslaweq, I think that ovs-agent egress-packet-flood issue should be target on U-1, lots of users have encountered that issue, what do you think?14:08
slaweqU-1 is this week so I don't think we can fix it as fast14:09
liuyulong#link https://bugs.launchpad.net/neutron/+bug/173206714:09
openstackLaunchpad bug 1732067 in OpenStack Security Advisory "openvswitch firewall flows cause flooding on integration bridge" [Undecided,Incomplete]14:09
liuyulong#link https://bugs.launchpad.net/neutron/+bug/184162214:09
openstackLaunchpad bug 1732067 in OpenStack Security Advisory "duplicate for #1841622 openvswitch firewall flows cause flooding on integration bridge" [Undecided,Incomplete]14:09
liuyulongAnd I've updated the fix: https://review.opendev.org/#/c/666991/14:09
liuyulongMy fault, haha14:10
slaweqliuyulong: Your fix is this fix which don't work for some dvr cases, do I remember that correctly from Shanghai?14:11
liuyulongHA router mixed with compute service14:12
liuyulongSince we have normal flows, and the qg-device can be hosted in HA-router scheduled nodes.14:13
slaweqok, I will review that patch asap14:14
liuyulongSo should I change the Milestone of the bug?14:15
*** njohnston_ has joined #openstack-meeting14:17
slaweqI will set it14:17
*** njohnston has quit IRC14:17
liuyulongGreat14:18
slaweqdone14:18
liuyulongSo IMO, if there are some more technical issues, we may mark them all. So the end users, may find something more easily when they want to find some thing useful or backporting something.14:19
*** ociuhandu_ has quit IRC14:20
liuyulongI will try to make that fix 666991 light to make it easy to backport.14:21
liuyulongOK, let's move on14:21
liuyulong#topic Bugs14:21
*** openstack changes topic to "Bugs (Meeting topic: neutron_l3)"14:21
liuyulongNo bug deputy mail this week.14:21
liuyulongSo let's scan the list.14:22
liuyulong#link https://bugs.launchpad.net/neutron/+bug/185591914:22
openstackLaunchpad bug 1855919 in neutron "broken pipe errors cause neutron metadata agent to fail" [Undecided,New]14:22
ralonsohI don't think (without having any other log), that this is a problem with the DB14:23
ralonsohbut with the MQ14:23
ralonsohcould be useful to debug the MQ status and to have the server logs14:24
liuyulongFrom my experiences, if the RPC client did not send the heart beat to the rabbit-server, rabbit-server will close the connection.14:24
liuyulongBut the client still consider that the connection is alive.14:24
ralonsohso do you think this is related to https://bugs.launchpad.net/neutron/+bug/1853071?14:25
openstackLaunchpad bug 1853071 in neutron "AMQP disconnects, q-reports-plugin queue grows, leading to DBDeadlocks while trying to update agent heartbeats" [High,In progress] - Assigned to Arjun Baindur (abaindur)14:25
ralonsohas suggested14:25
liuyulongSo the broken-pipe errors come.14:25
slaweqbut those errors "Timeout in RPC method get_ports" could be also caused by some issue on neutron-server side14:25
slaweqmaybe it's overloaded simple14:25
ralonsohsure14:25
*** njohnston_ is now known as njohnston14:25
slaweqwe should ask for neutron-server logs from the same time too14:25
ralonsohthat's why it should be useful to have both MQ and server logs14:25
slaweqralonsoh++14:25
liuyulongSome time, the long loop (without sleep 0) in the agent will cause the coroutines not switching. Then the heart beat greenlet will do nothing.14:27
liuyulongOne sec14:28
slaweqI added comment there and I set this bug as incomplete for now14:28
slaweqlets wait for additional data from reporter14:28
*** jamesmcarthur has joined #openstack-meeting14:31
liuyulongSure14:32
liuyulongThere are some config options from oslo_messaging, like heartbeat_interval, can be tuned for some heavy load use case.14:34
liuyulongseems no more bugs related to this meeting.14:35
liuyulongAny other bugs?14:36
*** jamesmcarthur has quit IRC14:36
liuyulongOK, let's move on.14:36
liuyulong#topic OVN_L314:38
*** openstack changes topic to "OVN_L3 (Meeting topic: neutron_l3)"14:38
*** ayoung has quit IRC14:38
liuyulongOne concern from me, last week we abandoned the floating IP QoS support patch for networking-ovn.14:40
liuyulong#link https://review.opendev.org/#/c/539826/14:40
liuyulongIs there any plan for this feature?14:41
slaweqI abandoned it automaticaly with script14:41
ralonsohonce we have it in Neutron repo, we can propose it again, but in Neutron14:41
slaweqif You or anyone else wants to continue work on this, we can always restore it14:41
liuyulongYes, it should be done in neutron.14:42
liuyulong#link https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge14:42
liuyulongWhat's the status of this now?14:42
ralonsohwe are still dealing with CI problems14:42
ralonsohbut is just a matter of rechecking14:42
liuyulongThere is a table I remember.14:42
ralonsohmost of the code is almost proposed in gerrit14:43
ralonsohand FTs are being testing right now14:43
ralonsohalong with the scripts needed for this14:43
ralonsohthat's all from me14:43
liuyulong#link https://review.opendev.org/#/q/topic:bp/neutron-ovn-merge+(status:open+OR+status:merged)14:44
*** jamesmcarthur has joined #openstack-meeting14:44
liuyulongThey are all here now. Only 5 open patches.14:45
ralonsohthe most important ones14:45
ralonsohthe ovsdb code, the mech driver and FTs14:45
ralonsoh(and trunk driver)14:45
liuyulongralonsoh, thank you for the information. : )14:46
ralonsohyw14:46
*** ociuhandu has joined #openstack-meeting14:46
liuyulongAlmost the entire core team is now on this BP. LOL14:47
*** eharney has quit IRC14:47
liuyulongOK, any updates for OVN l3?14:47
ralonsohnot from my side14:48
liuyulong#topic On demand agenda14:48
*** openstack changes topic to "On demand agenda (Meeting topic: neutron_l3)"14:48
*** links has quit IRC14:49
liuyulong_I have some interesting topics.14:49
liuyulong_Anyone who tried to run DVR with bare metal services?14:50
ralonsohno14:50
liuyulong_I mean your network is VLAN type, and you have physical hosts provisioned from ironic which also in such VLAN.14:51
liuyulong_So if you have many VM from many compute nodes, then your bare metal instances may have tons of gateways with same IP and MAC in these computes nodes.14:52
liuyulong_Another topic is to run DVR with L2 gateway services to connect the VXLAN network to the  bare metal switches?14:54
*** ociuhandu has quit IRC14:56
liuyulong_OK, I think this can be very interesting no matter neutron-agents or OVN try to make it come true.14:58
liuyulong_Time is up.14:59
liuyulong_#endmeeting14:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:59
openstackMeeting ended Wed Dec 11 14:59:49 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-12-11-13.59.html14:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-12-11-13.59.txt14:59
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-12-11-13.59.log.html14:59
ralonsohbye14:59
liuyulong_Bye guys.14:59
*** liuyulong_ has quit IRC15:00
*** liuyulong has quit IRC15:00
*** epei has joined #openstack-meeting15:00
slaweqbye15:00
*** epei has quit IRC15:05
*** jamesmcarthur has quit IRC15:06
*** ociuhandu has joined #openstack-meeting15:07
*** ociuhandu has quit IRC15:12
*** rsimai_away is now known as rsimai15:16
*** ayoung has joined #openstack-meeting15:21
*** eharney has joined #openstack-meeting15:23
*** ociuhandu has joined #openstack-meeting15:24
*** jamesmcarthur has joined #openstack-meeting15:25
*** jamesmcarthur_ has joined #openstack-meeting15:27
*** ociuhandu has quit IRC15:28
*** jamesmcarthur has quit IRC15:31
*** lpetrut has quit IRC15:32
*** links has joined #openstack-meeting15:38
*** ayoung has quit IRC15:39
*** panda is now known as panda|bbl15:44
*** ttsiouts has quit IRC15:48
*** Lucas_Gray has quit IRC15:54
*** Lucas_Gray has joined #openstack-meeting16:13
*** ayoung has joined #openstack-meeting16:15
*** rsimai is now known as rsimai_away16:23
*** ricolin has quit IRC16:46
*** guimaraes has quit IRC16:46
*** moguimar has joined #openstack-meeting16:47
*** moguimar has quit IRC16:47
*** gyee has joined #openstack-meeting16:55
*** LiangFang has quit IRC17:01
*** ayoung has quit IRC17:03
*** ayoung has joined #openstack-meeting17:03
*** dougwig has quit IRC17:08
*** dougwig has joined #openstack-meeting17:08
*** rpittau is now known as rpittau|afk17:13
*** psachin has quit IRC17:19
*** panda|bbl is now known as panda17:20
*** jlvillal has left #openstack-meeting17:20
*** links has quit IRC17:22
*** epei has joined #openstack-meeting17:29
*** ociuhandu has joined #openstack-meeting17:30
*** epei has quit IRC17:31
*** epei has joined #openstack-meeting17:36
*** epei has quit IRC17:41
*** igordc has joined #openstack-meeting17:42
*** diablo_rojo has joined #openstack-meeting17:44
*** ociuhandu has quit IRC17:45
*** epei has joined #openstack-meeting17:46
*** ayoung has quit IRC17:49
*** epei has quit IRC17:59
*** ijw has joined #openstack-meeting17:59
*** ijw_ has joined #openstack-meeting18:00
*** enriquetaso has joined #openstack-meeting18:01
*** ijw has quit IRC18:04
*** epei has joined #openstack-meeting18:04
*** epei has quit IRC18:09
*** ayoung has joined #openstack-meeting18:10
*** igordc has quit IRC18:13
*** igordc has joined #openstack-meeting18:14
*** e0ne has quit IRC18:14
*** epei has joined #openstack-meeting18:15
*** epei has quit IRC18:19
*** ociuhandu has joined #openstack-meeting18:24
*** epei has joined #openstack-meeting18:25
*** igordc has quit IRC18:25
*** enriquetaso has quit IRC18:28
*** ociuhandu has quit IRC18:29
*** jamesmcarthur_ has quit IRC18:30
*** epei has quit IRC18:30
*** epei has joined #openstack-meeting18:40
*** anastzhyr has quit IRC18:41
*** epei has quit IRC18:46
*** epei has joined #openstack-meeting18:51
*** enriquetaso has joined #openstack-meeting18:53
*** ralonsoh has quit IRC18:55
*** epei has quit IRC18:55
*** senrique_ has joined #openstack-meeting18:57
*** enriquetaso has quit IRC19:00
*** epei has joined #openstack-meeting19:02
*** epei has quit IRC19:06
*** epei has joined #openstack-meeting19:07
*** nicolasbock has quit IRC19:11
*** nicolasbock has joined #openstack-meeting19:11
*** epei has quit IRC19:12
*** epei has joined #openstack-meeting19:16
*** jamesmcarthur has joined #openstack-meeting19:20
*** eharney has quit IRC19:30
*** lpetrut has joined #openstack-meeting19:33
*** jamesmcarthur has quit IRC19:34
*** jamesmcarthur has joined #openstack-meeting19:34
*** Lucas_Gray has quit IRC19:35
*** mriedem has quit IRC19:37
*** ayoung has quit IRC19:41
*** tesseract has quit IRC19:46
*** lpetrut has quit IRC19:46
*** ayoung has joined #openstack-meeting20:02
*** lbragstad has joined #openstack-meeting20:03
*** epei has quit IRC20:03
*** epei has joined #openstack-meeting20:08
*** ociuhandu has joined #openstack-meeting20:09
*** eharney has joined #openstack-meeting20:12
*** epei has quit IRC20:13
*** ociuhandu has quit IRC20:21
*** ociuhandu has joined #openstack-meeting20:21
*** ociuhandu has quit IRC20:40
*** ociuhandu has joined #openstack-meeting20:41
*** ociuhandu has quit IRC20:57
timburke#startmeeting swift21:00
openstackMeeting started Wed Dec 11 21:00:09 2019 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: swift)"21:00
openstackThe meeting name has been set to 'swift'21:00
timburkewho's here for the swift meeting?21:00
mattoliverauo/21:00
kota_o/21:00
*** patchbot has joined #openstack-meeting21:00
seongsoochoo/21:00
rledisezo/21:00
tdasilvao/21:00
timburkehi everybody!21:01
timburkeagenda's at21:01
timburke#link https://wiki.openstack.org/wiki/Meetings/Swift21:01
timburke#topic V release naming21:01
*** openstack changes topic to "V release naming (Meeting topic: swift)"21:01
timburke#link http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011477.html21:01
timburkepolling's open!21:02
timburkehelp us name our next release :-)21:02
timburke#link https://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_13ccd49b66cfd1b4&akey=39d9234da798e6a321:02
timburkenote that "Vidette" was accidentally included twice -- you might want to ensure that both entries have the same ranking when submitting to make life easier for the election officials21:02
kota_lol21:03
*** slaweq has quit IRC21:03
timburkenot much more than that; mainly just wanted to make sure everyone was aware and could let their voice be heard :-)21:03
timburke#topic Keystone credentials API21:03
*** openstack changes topic to "Keystone credentials API (Meeting topic: swift)"21:04
timburkein light of the interest i heard in Shanghai about getting credential API support into swiftclient, i figured i ought to mention a recent security issue...21:04
timburke#link http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011529.html21:04
timburkeFWIW, we *do* use the credentials API for secret caching in s3token -- i haven't yet checked to see what sorts of recommendations we may need to make to ensure that continues working...21:05
timburkesee also...21:06
timburke#link https://github.com/openstack/swift/commit/b0aea936021:06
timburkemore broadly, this very much still seems like a thing swiftclient should support, though i don't know of anyone actively trying to implement it21:07
timburkebut i've also heard from some of *my* customers that they'd be interested, so... maybe i'll end up with some bandwidth for it?21:07
timburkeagain, wanted to raise awareness as much as anything -- if you *do* have customers using keystone application credentials, you might want to get them patched sooner rather than later21:09
timburke#topic end of year meetings21:09
*** openstack changes topic to "end of year meetings (Meeting topic: swift)"21:09
timburkewith the holidays coming up, i figured we ought to look at which scheduled meetings it makes sense to keep21:10
timburkeas things are, we'd hit both christmas (Dec 25) and new years (Jan 1)21:10
mattoliveraumakes sense, and yeah 25th and 1st make sense21:10
timburkebut i don't think it makes any sense to actually hold meetings on those days21:11
mattoliverauhit both the major holidays, wtg :P21:11
*** slaweq has joined #openstack-meeting21:11
timburkeanyone feel otherwise? *i* certainly won't be able to host, so any dissenters would be on the hook to chair the meeting ;-)21:11
* rledisez agrees to skip both of them21:11
tdasilva+121:11
timburkei *think* we've still got enough going on that it makes sense to keep our meeting for next week, though21:12
timburkeplus, it'll give me another opportunity to mention the skipped meetings ;-)21:13
*** ociuhandu has joined #openstack-meeting21:13
timburke(sorry to kota_ (and maybe others?) about the late notice for skipping the meeting a couple weeks ago)21:13
timburkeall right, seems like we're all on the same page :-)f21:15
timburkei'll also make sure i update the "next meeting" date on the wiki right after the meeting next week21:15
*** slaweq has quit IRC21:16
timburkeon to updates!21:16
timburke#topic versioning21:16
*** openstack changes topic to "versioning (Meeting topic: swift)"21:16
timburkeclayg, tdasilva -- how's it going?21:16
tdasilvaI applied some changes to the swift versioning patch early in the week based on reviews, now currently working on a container-sync patch21:17
tdasilvaso that container-sync will skip versioned containers (until we have a better solution for it)21:17
tdasilvathe s3 versioning is also in good shape, i noticed timburke put up a change recently to make sure s3api behaves correctly if versioning is not enabled i think, still need to look at that21:19
tdasilvamy hope is to see these patches merged by next week (fingers crossed)21:19
timburke(sounds like an action item for all of us: think about what you'd like container-sync to do with new-style versioned containers)21:20
timburkei did! pretty sure most everything works like it did before 🤞21:20
tdasilvai think that's all i got, any questions?21:21
tdasilvacontainer-sync patch without tests: https://review.opendev.org/#/c/698139/21:22
patchbotpatch 698139 - swift - WIP: prevent cont-sync when versioning enabled - 2 patch sets21:22
tdasilvain case you want to see the direction it is going21:22
timburkeif i remember right, there were also some questions about whether we'd want the new versioning enabled by default, or maybe whether we would want to *switch it* to be on-by-default in the near-ish future -- i'd certainly be interested in other people's thoughts on *that* question21:23
*** rcernin has joined #openstack-meeting21:23
mattoliverauneed to think about that.. but it is a different api.. and we want people to adopt it. Beside who wouldn't want to use versioning ;)21:24
tdasilvayeah, i think there are two sides to consider. one is what mattoliverau said + s3api versioning depends on it. OTOH, it does make use of the new reserved name API which is very very new21:25
mattoliverautest it with fire :P (but yeah that is the otherside)21:25
tdasilvawhat could go wrong21:26
timburkeit sure *feels like* something we'd want to have on-by-default eventually -- seems (to me, anyway) in all ways better to the versioning scheme we had before. so i guess, what sort of proof-points would people like to see before we move to on-by-default?21:27
timburkemake sure the api works as intended (ie, we don't go creating some objects in the null namespace that can't be deleted through some client-facing api), make sure we play well with at least container-sync...21:28
timburkeanything else?21:28
timburkeprobably look at expirer/reconciler interactions...21:28
mattoliveraumake sure sharding can shard null continers. I assume it can communicate with nulls. ie making null shards21:29
mattoliverauI assume this may have been already tested21:29
mattoliveraubut also assum versioned containers will potentually grow large over time21:30
tdasilvai think we looked at some probe tests for that...let me look21:30
* timburke is not sure that's a good assumption...21:30
*** raildo has quit IRC21:31
tdasilvaheh, maybe not21:32
mattoliverauI'll give it a go then :)21:32
timburkei suppose something like AWS's lifecycle management api would certainly be nice to help limit the growth of old versions... but since users would have to opt-in to enabling versioning, i'm not sure it should be a hard requirement that we have that before making it available by default...21:32
timburkeanyway, something to think about. i don't think we're likely to enable-by-default before the next time we all see each other face-to-face, anyway ;-)21:33
timburke#topic lots of files21:33
*** openstack changes topic to "lots of files (Meeting topic: swift)"21:33
*** lbragstad has quit IRC21:34
timburkerledisez, is there much new going on with this investigation?21:34
timburke(i was debating about whether to keep it on the agenda or drop it for now and let you add it back when there's new information to share)21:35
rledisezso alecuyer if not here today. he's currently working on some other ovh related things, for the next 3 or 4 weeks. next step for himm will be the battle zfs vs leveldb. so nothing really new for now21:35
timburke👍21:35
kota_ic21:36
rledisezi think you can drop it for now, we'll see in january when things start moving again21:36
timburkesounds good21:36
timburke#topic profiling21:36
*** openstack changes topic to "profiling (Meeting topic: swift)"21:36
timburkei saw a watchdog patch recently...21:36
rledisez#link https://review.opendev.org/#/c/697653/21:36
patchbotpatch 697653 - swift - Replace all "with Chunk*Timeout" by a watchdog - 5 patch sets21:36
*** mattw4 has joined #openstack-meeting21:36
timburkethat's the one :-)21:36
rledisezthere is some numbers in a comment that give an idea of the improvement21:37
rledisezafter that, i'll have an other patch that brings +30% download speed on EC (by removing greenthread/queue in reading fragments from object-servers), but I havce weird issues that, I'm now convinced, are related to greenthread scheduling & co… i don't know how i'll handle that21:38
rledisez(weird issues in unit tests only)21:38
*** pcaruana has quit IRC21:38
mattoliverauwow, nice!21:39
mattoliverauOnce I finish reviewing the versions patch, I look forward to taking these for a spin :)21:39
rledisezafter that i'll have some small patches that bring small improvement (caching some stuff, etc…)21:39
rledisezbut the bug win is in the patch linked here and the next one21:40
rledisez*big21:40
timburkesounds great :D21:40
timburkedoes anyone have any questions about the watchdog patch? i must admit, i haven't looked at it yet21:40
rlediseztdasilva: put some in the reviews. thx for looking at it, I hope i answered them :)21:41
*** epei has joined #openstack-meeting21:41
timburkewell, with three more items, maybe i'll keep it moving then ;-)21:42
timburke#topic MD5 sums21:42
*** openstack changes topic to "MD5 sums (Meeting topic: swift)"21:42
tdasilvarledisez: I did a quick scan of your comments, but TBH I need to look closer to make sure I understand it well21:42
timburke#link https://etherpad.openstack.org/p/swift-md5-optimisation21:42
tdasilvabut thanks for responding quickly21:42
rledisezthat's a first draft of my thoughts on MD5. it's very preliminary work, but i'm interested in any feedbacks21:43
timburkeinteresting... i don't think i've heard of XXH before...21:45
*** epei has quit IRC21:46
tdasilvarledisez: fwiw, i think azure is using crc64...21:46
*** epei has joined #openstack-meeting21:47
rlediseztdasilva: good to know. i was wondering if crc32 is enought given the default max object size is 5g21:47
timburkei think i rather like the idea of breaking up the client-proxy validation (which is an API that will be hard to break) from proxy-object validation (where we have a lot more freedom)21:47
rlediseztdasilva: i did some research and it seems it could be good enough (but i don't have solid argument for that)21:48
*** ociuhandu has quit IRC21:48
tdasilvarledisez: yeah, i'm not sure either, their objects sizes are even smaller IIUC21:48
tdasilvarledisez: what's 'MD5 + CRC32c * 3' ?21:48
rlediseztdasilva: it's a simulation where we would do MD5 once (for the etag) then 3 times crc32c for crypto/ec (like it's done actually with MD5)21:49
rlediseztimburke: can you reformulate a bit, i'm not sure i totally got you21:49
*** ayoung has quit IRC21:49
rledisezyou prefer to break client-proxy than proxy-object?21:50
tdasilvarledisez: but we then ended with something worse than what we have now?21:50
rlediseztdasilva: no, right now we have MD5*4 (1.64Gbps). If we had MD5+CRC32c*3, we would have 5.65Gbps => so better :)21:50
*** epei_ has joined #openstack-meeting21:51
timburkeso we care about multiple hops: client to proxy server, proxy server to object server. part of the problem for EC is that the data sent to each object server has to be hashed separately, since we can't just use a client-provided etag21:51
timburkesimilar arguments with encryption on triple-replica, though the overhead isn't quite as bad21:51
*** epei has quit IRC21:51
*** ociuhandu has joined #openstack-meeting21:52
*** ayoung has joined #openstack-meeting21:53
mattoliverauI guess it's just a shame we have the md5 contract with the client (the etag). other then that we can seperate things. which is what timburke is saying in the nutshell I think :)21:54
timburkehaving something in the proxy be responsible for ensuring data integrity with the client while some sort of lighter-weight integrity check for the rest of the pipeline and backend requests seems like a solid win21:54
mattoliverau+121:54
kota_+121:54
rledisez+1, we are on the same page :)21:54
timburkethe upgrade may get interesting -- but presumably we could just have an option that you don't switch on until you've upgraded all the backend servers21:55
mattoliverauif an op ends up "providing" and alg or not, I wonder if we should start returning etag (md5) and etag-alg in the hopes one day to deprecate the first ;)21:56
rledisezyeah, that's probably the right solution. but if you're using ssync, you might degrade nicely to moving data with MD5 I guess? but rsync would be an issue21:56
*** jamesmcarthur has quit IRC21:56
rledisezmattoliverau: totally!21:56
kota_probably we need to add sort of Accept: headers for the change between proxy and objext21:56
*** epei_ has quit IRC21:57
timburkeall right, running low on time. there were just a couple patched i wanted to call attention to21:57
timburke#topic ranged requests against SLOs21:57
*** openstack changes topic to "ranged requests against SLOs (Meeting topic: swift)"21:57
timburke#link https://review.opendev.org/#/c/697739/21:57
patchbotpatch 697739 - swift - Have slo tell the object-server that it wants whol... - 3 patch sets21:57
timburkeintroduces a new header, X-Backend-Ignore-Range-If-Metadata-Present, that let's SLO tell object servers that it wants the whole object if there's a X-Static-Large-Object header21:58
*** jamesmcarthur has joined #openstack-meeting21:58
timburkeotherwise, ranged SLO reads tend to look like:21:58
timburkeproxy issues GET with Range which 416s (because manifest is way smaller than large object)21:59
timburkeproxy issues GET, dropping Range, which gets manifest21:59
timburkeproxy issues GET to segment data21:59
timburkei want to get rid of that 416 back-and-forth22:00
mattoliverauoh i see22:00
timburke#topic RFC-compliant etags22:00
*** openstack changes topic to "RFC-compliant etags (Meeting topic: swift)"22:00
timburke#link https://review.opendev.org/#/c/695131/22:00
patchbotpatch 695131 - swift - Add proxy-server option to quote-wrap all ETags - 5 patch sets22:00
timburkei dusted off a 6-year-old Won't Fix bug :{22:00
timburkeer, :P22:00
mattoliveraulol22:00
*** dviroel has quit IRC22:00
timburkebut i'm also out of time ;-)22:00
timburkethank you all for coming today, and thank you for working on swift!22:01
timburke#endmeeting22:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:01
openstackMeeting ended Wed Dec 11 22:01:15 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-12-11-21.00.html22:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-12-11-21.00.txt22:01
mattoliverautimburke: make sure there on priority reviews so we can find them.. sorry if they already are :)22:01
openstackLog:            http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-12-11-21.00.log.html22:01
*** ayoung has quit IRC22:03
*** ayoung has joined #openstack-meeting22:04
*** ociuhandu has quit IRC22:06
*** ociuhandu has joined #openstack-meeting22:06
*** epei has joined #openstack-meeting22:07
*** ykatabam has joined #openstack-meeting22:11
*** slaweq has joined #openstack-meeting22:11
*** epei has quit IRC22:12
*** ykatabam has quit IRC22:13
*** slaweq has quit IRC22:16
*** epei has joined #openstack-meeting22:17
*** epei has quit IRC22:22
*** senrique__ has joined #openstack-meeting22:22
*** ykatabam has joined #openstack-meeting22:22
*** senrique_ has quit IRC22:22
*** senrique_ has joined #openstack-meeting22:24
*** rfolco has quit IRC22:25
*** senrique__ has quit IRC22:26
*** epei has joined #openstack-meeting22:27
*** ociuhandu has quit IRC22:27
*** nicolasbock has quit IRC22:27
*** epei has quit IRC22:31
*** epei has joined #openstack-meeting22:36
*** mriedem has joined #openstack-meeting22:37
*** epei has quit IRC22:41
*** rfolco has joined #openstack-meeting22:45
*** epei has joined #openstack-meeting22:45
*** epei has quit IRC22:50
*** jamesmcarthur has quit IRC22:56
*** epei has joined #openstack-meeting22:56
*** jamesmcarthur has joined #openstack-meeting23:03
*** jamesmcarthur has quit IRC23:03
*** epei has quit IRC23:06
*** epei has joined #openstack-meeting23:11
*** slaweq has joined #openstack-meeting23:11
*** epei has quit IRC23:15
*** slaweq has quit IRC23:15
*** Liang__ has joined #openstack-meeting23:20
*** epei has joined #openstack-meeting23:25
*** brault has quit IRC23:30
*** mmethot has quit IRC23:31
*** brault has joined #openstack-meeting23:31
*** epei has quit IRC23:31
*** slaweq has joined #openstack-meeting23:32
*** brault has quit IRC23:32
*** brault has joined #openstack-meeting23:33
*** epei has joined #openstack-meeting23:36
*** mmethot has joined #openstack-meeting23:36
*** epei has quit IRC23:41
*** slaweq has quit IRC23:41
*** Liang__ has quit IRC23:42
*** epei has joined #openstack-meeting23:46
*** epei has quit IRC23:51
*** epei has joined #openstack-meeting23:57
*** brinzhang has joined #openstack-meeting23:58
*** brault has quit IRC23:58
*** brault has joined #openstack-meeting23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!