Tuesday, 2018-06-26

*** bobh has joined #openstack-meeting00:00
*** pchavva has quit IRC00:02
*** heyongli has quit IRC00:03
*** heyongli has joined #openstack-meeting00:03
*** yamamoto has quit IRC00:03
*** shaohe_feng has quit IRC00:05
*** shaohe_feng has joined #openstack-meeting00:06
*** mjturek has quit IRC00:08
*** chyka has joined #openstack-meeting00:10
*** salv-orlando has joined #openstack-meeting00:11
*** heyongli has quit IRC00:13
*** heyongli has joined #openstack-meeting00:13
*** shaohe_feng has quit IRC00:15
*** chyka has quit IRC00:15
*** shaohe_feng has joined #openstack-meeting00:16
*** salv-orlando has quit IRC00:16
*** heyongli has quit IRC00:23
*** heyongli has joined #openstack-meeting00:23
*** tetsuro has quit IRC00:24
*** shaohe_feng has quit IRC00:25
*** tetsuro has joined #openstack-meeting00:26
*** shaohe_feng has joined #openstack-meeting00:26
*** aeng has quit IRC00:33
*** heyongli has quit IRC00:33
*** heyongli has joined #openstack-meeting00:34
*** kbyrne has quit IRC00:34
*** Swami_ has quit IRC00:34
*** Swami has quit IRC00:34
*** kbyrne has joined #openstack-meeting00:35
*** shaohe_feng has quit IRC00:35
*** edmondsw has quit IRC00:37
*** longkb has joined #openstack-meeting00:38
*** shaohe_feng has joined #openstack-meeting00:40
*** heyongli has quit IRC00:44
*** heyongli has joined #openstack-meeting00:44
*** abelur_ has joined #openstack-meeting00:45
*** shaohe_feng has quit IRC00:46
*** aeng has joined #openstack-meeting00:46
*** abelur_ has quit IRC00:48
*** shaohe_feng has joined #openstack-meeting00:48
*** wanghao_ has quit IRC00:52
*** wanghao has joined #openstack-meeting00:52
*** heyongli has quit IRC00:54
*** heyongli has joined #openstack-meeting00:54
*** shaohe_feng has quit IRC00:56
*** shaohe_feng has joined #openstack-meeting00:58
*** markstur has quit IRC00:58
*** boden has joined #openstack-meeting00:58
*** yamamoto has joined #openstack-meeting00:59
*** boden has quit IRC01:03
*** heyongli has quit IRC01:04
*** njohnston has quit IRC01:04
*** heyongli has joined #openstack-meeting01:04
*** yamamoto has quit IRC01:05
*** shaohe_feng has quit IRC01:06
*** shaohe_feng has joined #openstack-meeting01:07
*** salv-orlando has joined #openstack-meeting01:11
*** ekcs has quit IRC01:12
*** heyongli has quit IRC01:14
*** heyongli has joined #openstack-meeting01:15
*** kaisers_ has joined #openstack-meeting01:15
*** salv-orlando has quit IRC01:15
*** shaohe_feng has quit IRC01:16
*** shaohe_feng has joined #openstack-meeting01:18
*** kaisers has quit IRC01:19
*** hongbin has joined #openstack-meeting01:21
*** heyongli has quit IRC01:25
*** heyongli has joined #openstack-meeting01:25
*** shaohe_feng has quit IRC01:27
*** shaohe_feng has joined #openstack-meeting01:28
*** fzdarsky_ has joined #openstack-meeting01:28
*** fzdarsky has quit IRC01:32
*** imacdonn has joined #openstack-meeting01:35
*** heyongli has quit IRC01:35
*** heyongli has joined #openstack-meeting01:35
*** yangyapeng has joined #openstack-meeting01:36
*** shaohe_feng has quit IRC01:37
*** bobh has quit IRC01:37
*** shaohe_feng has joined #openstack-meeting01:40
*** hemna_ has quit IRC01:41
*** gongysh has joined #openstack-meeting01:42
*** julim has joined #openstack-meeting01:44
*** heyongli has quit IRC01:45
*** heyongli has joined #openstack-meeting01:45
*** harlowja has quit IRC01:46
*** shaohe_feng has quit IRC01:47
*** gongysh has quit IRC01:47
*** mordred has quit IRC01:49
*** shaohe_feng has joined #openstack-meeting01:49
*** yamahata has quit IRC01:51
*** heyongli has quit IRC01:55
*** jamesmcarthur has joined #openstack-meeting01:55
*** heyongli has joined #openstack-meeting01:56
*** shaohe_feng has quit IRC01:57
*** shaohe_feng has joined #openstack-meeting01:59
*** jamesmcarthur has quit IRC02:00
*** heyongli has quit IRC02:06
*** heyongli has joined #openstack-meeting02:06
*** mordred has joined #openstack-meeting02:07
*** shaohe_feng has quit IRC02:08
*** shaohe_feng has joined #openstack-meeting02:08
*** heyongli has quit IRC02:16
*** heyongli has joined #openstack-meeting02:16
*** shaohe_feng has quit IRC02:18
*** shaohe_feng has joined #openstack-meeting02:20
*** iyamahat_ has quit IRC02:23
*** heyongli has quit IRC02:26
*** heyongli has joined #openstack-meeting02:26
*** shaohe_feng has quit IRC02:28
*** shaohe_feng has joined #openstack-meeting02:29
*** heyongli has quit IRC02:36
*** heyongli has joined #openstack-meeting02:37
*** shaohe_feng has quit IRC02:38
*** shaohe_feng has joined #openstack-meeting02:40
*** psachin` has joined #openstack-meeting02:42
*** ricolin has joined #openstack-meeting02:42
*** bkopilov has quit IRC02:43
*** heyongli has quit IRC02:47
*** heyongli has joined #openstack-meeting02:47
*** shaohe_feng has quit IRC02:49
*** shaohe_feng has joined #openstack-meeting02:50
*** bkopilov has joined #openstack-meeting02:57
*** heyongli has quit IRC02:57
*** heyongli has joined #openstack-meeting02:57
*** tpatil has joined #openstack-meeting02:58
*** shaohe_feng has quit IRC02:59
tpatilsamP: Ping?03:00
*** shaohe_feng has joined #openstack-meeting03:01
tpatil#startmeeting Masakari03:01
openstackMeeting started Tue Jun 26 03:01:14 2018 UTC and is due to finish in 60 minutes.  The chair is tpatil. Information about MeetBot at http://wiki.debian.org/MeetBot.03:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.03:01
*** openstack changes topic to " (Meeting topic: Masakari)"03:01
openstackThe meeting name has been set to 'masakari'03:01
tpatilHi All03:01
samPsorry Im late03:01
*** yamamoto has joined #openstack-meeting03:01
samPHi03:01
tpatilHi samP03:02
*** Dinesh_Bhor has joined #openstack-meeting03:02
tpatilPlease take the pilot seat03:02
samPnot started yet, right?03:02
samPtpatil: sure..03:02
tpatilalready started03:02
samPah.. yes thanks03:02
samPOK then,03:02
samP#topic bugs and patches03:03
samPany bugs or patches need to discuss?03:04
tpatilLouie Kwan has uploaded a new PS03:04
tpatilI'm busy reviewing other patches. Need help in reviewing his patch03:04
tpatil#link https://review.openstack.org/#/c/534958/03:04
patchbotpatch 534958 - masakari-monitors - Introspective Instance Monitoring through QEMU Gue...03:04
samPtpatil: great.,, I will take a look on that03:04
samPtpatil: thanks for the review03:05
tpatilThank you03:05
tpatilI have proposed a fix for LP bug https://bugs.launchpad.net/masakari/+bug/177313203:05
openstackLaunchpad bug 1773132 in masakari "masakari-engine runs recovery twice for one notification when disconnection with rabbitmq" [Undecided,In progress] - Assigned to Tushar Patil (tpatil)03:05
tpatil#link https://review.openstack.org/#/c/576042/03:05
patchbotpatch 576042 - masakari - Avoid recovery from failure twice03:05
tpatilPlease take a look at this patch as well03:05
samPtpatil: sure03:06
samPtpatil: thanks for the fix03:06
*** heyongli has quit IRC03:07
*** yamamoto has quit IRC03:07
*** heyongli has joined #openstack-meeting03:07
tpatilOther LP bug https://bugs.launchpad.net/masakari/+bug/177376503:08
openstackLaunchpad bug 1773765 in masakari "There is a possibility that 'running' notification will remain" [Undecided,New]03:08
tpatilIt is little tricky to fix this issue becoz we don't have much information about which instances are already evacuated03:08
tpatilMasakari doesn't save any info about which instances are evacuated and how many are still pending during host failure notification03:09
tpatilhence it's difficult to re-execute running status notification03:09
*** shaohe_feng has quit IRC03:09
samPhumm...I think we can store some of these info in masakari, and with current status form nova, we are able to calculate it.03:11
samPBut, not sure is this the best way to deal with.03:11
*** shaohe_feng has joined #openstack-meeting03:11
tpatilyes, it's a big change03:12
tpatilCan we focus on this issue in next release?03:12
tpatilIf we decide to maintain basic info in masakari, we can also add code to show the current progress of host failure notification03:13
tpatillike source host failure, out of 10 instances, 5th instance evacuation is in progress03:13
samPtpatil: Indeed.03:13
samPtoabctl: IIRC, nova does have api for get the status for migration,03:14
tpatilsamP: Yes, but Masakari will still need to maintain basic info to query status from nova03:17
*** heyongli has quit IRC03:17
*** heyongli has joined #openstack-meeting03:18
samPtpatil: agree. In any cause, it seems like this fix is too heavy for this release.03:18
tpatilsamP: Correct03:18
samPLet's postpone it to next release. Otherwise, some one raise urgent flag..!03:19
*** shaohe_feng has quit IRC03:19
*** shaohe_feng has joined #openstack-meeting03:20
samPany other bugs or patches?03:20
tpatilOne minor issue03:21
tpatil#link https://review.openstack.org/#/c/574614/03:21
patchbotpatch 574614 - masakari - Segment description allows multiline characters03:21
tpatilDescription doesn't allow multiline characters03:21
tpatilI will review this patch today03:21
samPLGTM03:23
samPneed to rebase03:24
samPthanks for the fix03:24
tpatilOk03:24
tpatilI have voted + 2 on this patch https://review.openstack.org/#/c/567825/03:25
patchbotpatch 567825 - masakari - Enable mutable config in Masakari03:25
tpatilNeed someone to review and approve this patch03:25
samPsure, I will take a look03:26
samPtpatil: thanks03:26
samPLet's move to status update for features and sub-projects03:27
*** heyongli has quit IRC03:28
samP(1) Horizon Plugin03:28
*** heyongli has joined #openstack-meeting03:28
tpatilI have reviewed 3-4 patches and posted comments03:28
samPI want to review, but it will take some time to get there...;)03:29
tpatilok03:29
samPbut I will be their soon03:29
*** shaohe_feng has quit IRC03:30
samPtpatil: thanks for reviews03:30
samP(2) recovery method customization03:30
tpatilspecs to be reviewed03:31
samPI read the spec. I will put my comments soon, No critical comments, basically LGTM03:31
tpatilImplementation is almost finished. Working on adding unit tests. Expect to push patch in next week03:31
samPtpatil: sure, thanks a lot03:31
*** shaohe_feng has joined #openstack-meeting03:31
samP(3) Ansible support for Masakari03:32
samPAny issues?03:33
tpatilThis task is currently on hold due to unavailability of dev. environment03:33
tpatilwe had discussed about this issue in last-to-last meeting03:33
samPtpatil: ah, yes. my fault03:33
samPI will try to provide server init method asap03:34
tpatilStill many items are pending. Looks like we will need to postpone it to the next release03:34
tpatilsamP: Thanks03:34
samPtpatil: are you referring to ansible?03:36
tpatilyes03:36
*** chyka has joined #openstack-meeting03:36
samPtpatil: monitor part is bit difficult and I think it is OK to postpone it to next release03:37
*** heyongli has quit IRC03:38
*** yamahata has joined #openstack-meeting03:38
tpatilsamP: Agree03:38
*** heyongli has joined #openstack-meeting03:38
samPif some one came up with a quick solution, then we can reconsider to put it into to this release.03:39
tpatilok03:39
samPOtherwise, let's consider it on next release03:40
samP#topic AOB03:40
*** shaohe_feng has quit IRC03:40
tpatilI want to bring up this issue report by Torin on openstack ML03:40
tpatil#link http://lists.openstack.org/pipermail/openstack/2018-June/046620.html03:40
tpatils/report/reported03:40
*** shaohe_feng has joined #openstack-meeting03:41
*** psahoo has joined #openstack-meeting03:41
*** chyka has quit IRC03:41
tpatilexecute method is failing with 124 error03:41
samPhumm.... seems like Corosync faild to sync03:42
tpatiltimeout returns 124 error, but if you pass --preserve-status option to timeout it return 003:43
tpatilIn that case, I don't think it will raise ProcessExecutionError03:44
tpatilDo we expect host monitor process to be run by superuser?03:45
tpatilbecoz run_as_root=True is passed to utils.execute method03:46
tpatilhttps://github.com/openstack/masakari-monitors/blob/master/masakarimonitors/hostmonitor/host_handler/handle_host.py#L11303:46
samPtpatil: case here is, tcpdump can not be run by the normal user.03:47
*** heyongli has quit IRC03:48
*** heyongli has joined #openstack-meeting03:48
tpatilIf you have any info about this issue, can you please reply to the above ML thread03:49
samPtpatil: sure, I will join to discussion.03:49
samPI need to read the thread carefully. I will reply to thread03:49
*** lpetrut has joined #openstack-meeting03:50
samPtpatil: thanks for answering to thread03:50
*** shaohe_feng has quit IRC03:50
*** yangyapeng has quit IRC03:50
samPah, and this is openstack ML, not the dev03:51
*** yangyapeng has joined #openstack-meeting03:51
tpatilyes03:51
samPOK got it.03:51
tpatilAfter upgrading to Queens, Torin is not able to run host monitor process03:52
samPThat is critical.03:52
*** shu-mutow has joined #openstack-meeting03:52
*** shaohe_feng has joined #openstack-meeting03:52
samPtpatil: I will take a look on this issue.03:52
tpatilThanks03:53
samPAny other items to discuss?03:54
samPOtherwise, let finish today's meeting.03:54
tpatilThat's all from my end for now03:54
*** hongbin has quit IRC03:54
samPPlease use openstack-dev ML or IRC #openstack-masakari on freenode for further discussion..03:55
*** yangyapeng has quit IRC03:55
samPthank you for attending today's meeting03:55
samPtpatil: thanks03:55
samP#endmeeting03:55
tpatilThank you, Bye03:56
samPah, you have to end it..03:56
tpatilyes will do03:56
tpatil#endmeeting03:56
samPtpatil: :)03:56
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"03:56
openstackMeeting ended Tue Jun 26 03:56:22 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)03:56
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-06-26-03.01.html03:56
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-06-26-03.01.txt03:56
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-06-26-03.01.log.html03:56
samPtpatil: thanks03:56
*** tpatil has quit IRC03:56
*** heyongli has quit IRC03:58
*** heyongli has joined #openstack-meeting03:59
*** shaohe_feng has quit IRC04:00
*** shaohe_feng has joined #openstack-meeting04:04
*** yamamoto has joined #openstack-meeting04:04
*** heyongli has quit IRC04:09
*** heyongli has joined #openstack-meeting04:09
*** yamamoto has quit IRC04:09
*** shaohe_feng has quit IRC04:11
*** shaohe_feng has joined #openstack-meeting04:12
*** ekcs has joined #openstack-meeting04:14
*** sridharg has joined #openstack-meeting04:19
*** heyongli has quit IRC04:19
*** heyongli has joined #openstack-meeting04:19
*** shaohe_feng has quit IRC04:21
*** shaohe_feng has joined #openstack-meeting04:21
*** yangyapeng has joined #openstack-meeting04:26
*** lpetrut has quit IRC04:28
*** ekcs has quit IRC04:29
*** heyongli has quit IRC04:29
*** heyongli has joined #openstack-meeting04:29
*** dims has quit IRC04:30
*** shaohe_feng has quit IRC04:31
*** ekcs has joined #openstack-meeting04:33
*** shaohe_feng has joined #openstack-meeting04:33
*** dims has joined #openstack-meeting04:35
*** psahoo has quit IRC04:35
*** diablo_rojo has quit IRC04:37
*** heyongli has quit IRC04:39
*** heyongli has joined #openstack-meeting04:40
*** shaohe_feng has quit IRC04:41
*** shaohe_feng has joined #openstack-meeting04:42
*** cloudrancher has joined #openstack-meeting04:47
*** heyongli has quit IRC04:50
*** heyongli has joined #openstack-meeting04:50
*** psahoo has joined #openstack-meeting04:50
*** shaohe_feng has quit IRC04:52
*** shaohe_feng has joined #openstack-meeting04:53
*** janki has joined #openstack-meeting04:54
*** heyongli has quit IRC05:00
*** heyongli has joined #openstack-meeting05:00
*** armax has joined #openstack-meeting05:01
*** shaohe_feng has quit IRC05:02
*** shaohe_feng has joined #openstack-meeting05:03
*** aeng has quit IRC05:03
*** yamamoto has joined #openstack-meeting05:05
*** ekcs has quit IRC05:06
*** ekcs has joined #openstack-meeting05:07
*** slaweq has joined #openstack-meeting05:07
*** slaweq has quit IRC05:07
*** jchhatbar has joined #openstack-meeting05:09
*** heyongli has quit IRC05:10
*** heyongli has joined #openstack-meeting05:10
*** numans has quit IRC05:11
*** shaohe_feng has quit IRC05:12
*** yamamoto has quit IRC05:12
*** numans has joined #openstack-meeting05:13
*** janki has quit IRC05:13
*** e0ne has joined #openstack-meeting05:13
*** shaohe_feng has joined #openstack-meeting05:13
*** chyka has joined #openstack-meeting05:14
*** cloudrancher has quit IRC05:15
*** lifeless has joined #openstack-meeting05:16
*** vabada2 has joined #openstack-meeting05:16
*** ekcs has quit IRC05:16
*** arnewiebalck_ has joined #openstack-meeting05:16
*** strigazi has quit IRC05:17
*** vabada has quit IRC05:18
*** arnewiebalck has quit IRC05:18
*** arnewiebalck_ is now known as arnewiebalck05:18
*** chyka has quit IRC05:19
*** strigazi has joined #openstack-meeting05:19
*** heyongli has quit IRC05:20
*** heyongli has joined #openstack-meeting05:21
*** shaohe_feng has quit IRC05:22
*** arnewiebalck_ has joined #openstack-meeting05:24
*** vabada has joined #openstack-meeting05:24
*** strigazi_ has joined #openstack-meeting05:24
*** shaohe_feng has joined #openstack-meeting05:25
*** arnewiebalck has quit IRC05:26
*** arnewiebalck_ is now known as arnewiebalck05:26
*** strigazi has quit IRC05:27
*** vabada2 has quit IRC05:27
*** heyongli has quit IRC05:31
*** heyongli has joined #openstack-meeting05:31
*** armax has quit IRC05:32
*** shaohe_feng has quit IRC05:33
*** shaohe_feng has joined #openstack-meeting05:33
*** armax has joined #openstack-meeting05:34
*** psahoo has quit IRC05:36
*** heyongli has quit IRC05:41
*** heyongli has joined #openstack-meeting05:41
*** shaohe_feng has quit IRC05:43
*** Dinesh_Bhor has quit IRC05:44
*** shaohe_feng has joined #openstack-meeting05:44
*** jmnk has joined #openstack-meeting05:49
*** heyongli has quit IRC05:51
*** heyongli has joined #openstack-meeting05:51
*** psahoo has joined #openstack-meeting05:51
*** shaohe_feng has quit IRC05:53
*** shaohe_feng has joined #openstack-meeting05:55
*** tssurya has joined #openstack-meeting05:56
*** heyongli has quit IRC06:01
*** heyongli has joined #openstack-meeting06:02
*** shaohe_feng has quit IRC06:03
*** shaohe_feng has joined #openstack-meeting06:06
*** e0ne has quit IRC06:08
*** yamamoto has joined #openstack-meeting06:09
*** slaweq has joined #openstack-meeting06:10
*** heyongli has quit IRC06:12
*** heyongli has joined #openstack-meeting06:12
*** shaohe_feng has quit IRC06:14
*** yamamoto has quit IRC06:14
*** shaohe_feng has joined #openstack-meeting06:17
*** alexchadin has joined #openstack-meeting06:20
*** heyongli has quit IRC06:22
*** heyongli has joined #openstack-meeting06:22
*** cloudrancher has joined #openstack-meeting06:24
*** shaohe_feng has quit IRC06:24
*** shaohe_feng has joined #openstack-meeting06:25
*** heyongli has quit IRC06:32
*** heyongli has joined #openstack-meeting06:32
*** andreas_s has joined #openstack-meeting06:33
*** qwebirc7707 has joined #openstack-meeting06:34
*** rbartal has joined #openstack-meeting06:34
*** shaohe_feng has quit IRC06:34
*** shaohe_feng has joined #openstack-meeting06:36
*** qwebirc7707 has quit IRC06:38
*** heyongli has quit IRC06:42
*** heyongli has joined #openstack-meeting06:43
*** shaohe_feng has quit IRC06:44
*** psachin` has quit IRC06:45
*** shaohe_feng has joined #openstack-meeting06:45
*** psachin` has joined #openstack-meeting06:47
*** peereb has joined #openstack-meeting06:48
*** peereb has quit IRC06:49
*** peereb has joined #openstack-meeting06:49
*** peereb has quit IRC06:50
*** alexchadin has quit IRC06:52
*** heyongli has quit IRC06:53
*** heyongli has joined #openstack-meeting06:53
*** shaohe_feng has quit IRC06:55
*** caobin has joined #openstack-meeting06:55
*** yamamoto has joined #openstack-meeting06:56
*** shaohe_feng has joined #openstack-meeting06:56
*** ircuser-1 has joined #openstack-meeting06:58
*** heyongli has quit IRC07:03
*** heyongli has joined #openstack-meeting07:03
*** shaohe_feng has quit IRC07:05
*** shaohe_feng has joined #openstack-meeting07:06
*** slaweq has quit IRC07:06
*** lpetrut has joined #openstack-meeting07:07
*** rcernin has quit IRC07:08
*** slaweq has joined #openstack-meeting07:10
*** gongysh has joined #openstack-meeting07:10
*** caobin has quit IRC07:12
*** caobin has joined #openstack-meeting07:13
*** heyongli has quit IRC07:13
*** heyongli has joined #openstack-meeting07:13
*** tesseract has joined #openstack-meeting07:14
*** shaohe_feng has quit IRC07:15
*** slaweq has quit IRC07:15
*** shaohe_feng has joined #openstack-meeting07:17
*** pcaruana has joined #openstack-meeting07:20
*** kopecmartin has joined #openstack-meeting07:23
*** heyongli has quit IRC07:23
*** heyongli has joined #openstack-meeting07:24
*** andreas_s has quit IRC07:25
*** shaohe_feng has quit IRC07:25
*** andreas_s has joined #openstack-meeting07:26
*** shaohe_feng has joined #openstack-meeting07:26
*** andreas_s_ has joined #openstack-meeting07:28
*** yamahata has quit IRC07:30
*** andreas_s has quit IRC07:32
*** heyongli has quit IRC07:34
*** heyongli has joined #openstack-meeting07:34
*** shaohe_feng has quit IRC07:36
*** shaohe_feng has joined #openstack-meeting07:36
*** phil_ has joined #openstack-meeting07:37
*** phil_ is now known as Guest160607:37
*** andreas_s_ has quit IRC07:38
*** andreas_s has joined #openstack-meeting07:39
*** andreas_s has quit IRC07:39
*** andreas_s_ has joined #openstack-meeting07:40
*** AndreyS has joined #openstack-meeting07:42
*** cloudrancher has quit IRC07:43
*** heyongli has quit IRC07:44
*** heyongli has joined #openstack-meeting07:44
*** ahrechny has joined #openstack-meeting07:45
*** Magistrazh has joined #openstack-meeting07:45
*** Magistrazh has left #openstack-meeting07:45
*** akozhevnikov has joined #openstack-meeting07:46
*** shaohe_feng has quit IRC07:46
*** shaohe_feng has joined #openstack-meeting07:47
*** phuoc has joined #openstack-meeting07:48
*** ttsiouts has joined #openstack-meeting07:51
*** heyongli has quit IRC07:54
*** heyongli has joined #openstack-meeting07:54
*** joxyuki has joined #openstack-meeting07:56
*** shaohe_feng has quit IRC07:56
*** dkushwaha has joined #openstack-meeting07:56
*** shaohe_feng has joined #openstack-meeting07:57
*** alexchadin has joined #openstack-meeting07:58
*** caobin has left #openstack-meeting07:58
*** andreas_s_ has quit IRC07:59
*** andreas_s has joined #openstack-meeting08:00
*** andreas_s has quit IRC08:00
*** andreas_s has joined #openstack-meeting08:01
*** alexchadin has quit IRC08:01
gongysh#startmeeting tacker08:02
openstackMeeting started Tue Jun 26 08:02:07 2018 UTC and is due to finish in 60 minutes.  The chair is gongysh. Information about MeetBot at http://wiki.debian.org/MeetBot.08:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:02
*** openstack changes topic to " (Meeting topic: tacker)"08:02
openstackThe meeting name has been set to 'tacker'08:02
gongysh#topic roll call08:02
*** openstack changes topic to "roll call (Meeting topic: tacker)"08:02
dkushwahao/08:02
gongyshdkushwaha, hi08:03
joxyukihi08:03
*** jamesmcarthur has joined #openstack-meeting08:03
phuochi08:03
*** heyongli has quit IRC08:04
*** slaweq has joined #openstack-meeting08:04
*** heyongli has joined #openstack-meeting08:05
*** e0ne has joined #openstack-meeting08:05
gongyshok08:05
gongyshit seems other guys are busy.08:05
gongysh#topic bp08:05
*** openstack changes topic to "bp (Meeting topic: tacker)"08:05
gongyshphuoc, any thing to talk?08:05
phuocgongysh, I havee some bug with devstack08:06
phuocbut I think it can be fixed in this week08:06
*** browny has quit IRC08:06
AndreySHi, everyone08:06
gongyshAndreyS, hi08:06
longkbo/08:06
*** shaohe_feng has quit IRC08:06
dkushwahaAndreyS, hi08:07
AndreySwe would like to discuss BP https://blueprints.launchpad.net/tacker/+spec/tosca-csar-mgmt-driver08:07
gongyshAndreyS, can you introduce yourself first?08:07
AndreySAndrey Sviridov08:08
*** cloudrancher has joined #openstack-meeting08:08
phuocAndreyS, nice to see you :)08:08
*** alexchadin has joined #openstack-meeting08:08
*** jamesmcarthur has quit IRC08:08
gongyshAndreyS,  what is your company?08:08
ahrechnyHi all, I work with AndreyS on that blueprint and have some questions08:08
*** Li_Jiale has joined #openstack-meeting08:09
ahrechnyWe are from Netcracker/DTAG08:09
AndreySMy company is Netcracker08:09
joxyukiAndreyS, ahrechny, hi08:09
gongyshahrechny, hi, welcome08:09
phuocwelcome to Tacker :)08:10
gongyshAndreyS, ok do you have any idea for that spec?08:10
ahrechnyWe want to add ability to support csar packages only for VNFM08:10
akozhevnikovHi all, also working with AndreyS and ahrechny08:10
gongyshakozhevnikov, hi08:10
gongyshbig welcome08:10
ahrechnyAnd want to discuss some architecture points08:11
gongyshahrechny, ok08:11
gongysh go on08:11
nguyenhaio/08:11
ahrechnyFirst of all - rest api08:11
ahrechnynow it supports only text descriptors08:11
ahrechnybut we need to upload files08:12
*** shaohe_feng has joined #openstack-meeting08:12
gongyshahrechny, yes08:12
ahrechnyso we have some ways for it - send serialized text or add new api for files uploading08:13
AndreySI agree with ahrecny that ability to onboard CSAR VNF  packages (not simple yaml, but archive) is very important08:13
ahrechnyis it possible to change current API?08:13
*** caobin has joined #openstack-meeting08:14
caobinhello08:14
gongyshahrechny you can have a draft design in a spec file08:14
gongyshcaobin, helo08:14
gongyshwelcome08:14
gongyshcaobin, could you please introduce yourself first?08:14
Li_JialeMy colleague08:14
*** heyongli has quit IRC08:15
ahrechnyunfortunately we don't have draft yet08:15
gongyshcaobin, welcome.08:15
*** heyongli has joined #openstack-meeting08:15
caobinthanks08:15
*** slaweq has quit IRC08:15
gongyshahrechny, to save csar files on tacker server side, we can use ceph.08:15
gongyshahrechny,  please have draft design, and then we can have a talk at the next meeting.08:16
*** cloudrancher has quit IRC08:16
*** shaohe_feng has quit IRC08:17
phuocI think you guys want to upload csar file to Tacker server08:17
*** cloudrancher has joined #openstack-meeting08:17
*** shaohe_feng has joined #openstack-meeting08:17
phuocthen Tacker server will create something, such as images, files08:18
ahrechnybut one important question about API - may I change current method for creating VNF or I should create new one?08:18
phuocand then create VNFD ?08:18
gongyshahrechny, I think csar is a way to create a vnfd, nsd, or vnffgd08:19
ahrechnywe want to create VNFD from csar, not from plain yaml file08:19
ahrechny*not only08:19
phuocahrechny, IMO we should extend current method08:19
dkushwahaahrechny, IMO, it can be done with existing api too, needs to check that08:20
ahrechnyBut it means that I should change content type08:20
ahrechnyfor example to multipart/form-data instead of application/json08:21
*** cloudrancher has quit IRC08:21
*** salv-orlando has joined #openstack-meeting08:22
gongyshcsar should be a self explained archive. it will read it is a vnfd, vnfgd or nsd something08:22
*** cloudrancher has joined #openstack-meeting08:22
ahrechnyyes, but we want to start for VNFM08:23
ahrechnythan it can be extended for NFVO08:23
gongyshahrechny, first you need to have an idea what is the csar. maybe to refer to some standards or other mano systems.08:23
gongyshand then we can see how the workflow is.08:24
ahrechnyI have some examples08:24
*** YanXingAn has joined #openstack-meeting08:24
akozhevnikovThis will also require changes in the taker-client, now there is only support for yaml when creating VNFD08:24
*** cloudrancher has quit IRC08:24
*** julim has quit IRC08:24
AndreySarchive for VNF descriptor, which contains yaml file, meta data, imports, definitions of types, artifacts & etc08:25
*** heyongli has quit IRC08:25
*** heyongli has joined #openstack-meeting08:25
gongyshAndreyS, I think you guys need a draft design, and we can have a talk.08:25
*** salv-orlando has quit IRC08:25
gongyshwho knows Dinesh Bhor ?08:26
*** salv-orlando has joined #openstack-meeting08:26
dkushwahagongysh, yea, I know08:26
joxyukigongysh, yes08:26
gongyshis he online?08:26
joxyukino08:27
*** kaisers1 has quit IRC08:27
gongyshhe just updated a patch08:27
*** shaohe_feng has quit IRC08:27
gongyshhttps://review.openstack.org/#/c/570845/08:27
patchbotpatch 570845 - tacker - Add placement policies support08:27
*** cloudrancher has joined #openstack-meeting08:27
*** shaohe_feng has joined #openstack-meeting08:28
gongyshhe also did not reply to the comment08:28
*** cloudrancher has quit IRC08:28
joxyukigongysh, do you want him to attend this weekly meeting?08:28
gongyshsure08:28
joxyukiok, I will talk to him to attend next meeting08:29
gongyshjoxyuki, thanks08:29
gongyshI want to merge the policy spec early08:29
gongyshsince we are close to rocky cycle.08:30
joxyukioh, I understood08:30
joxyukiI talk to him to join tacker channel08:30
gongyshYanXingAn, hi08:31
YanXingAnhi08:31
gongyshhow is the barbican problem?08:31
gongyshjoxyuki, if you have time, please have a test that patch too.08:32
gongyshit is almost done.08:32
gongyshthere is no big issue with that patch.08:32
joxyukihttps://review.openstack.org/#/c/570845/ ?08:32
patchbotpatch 570845 - tacker - Add placement policies support08:32
*** psahoo has quit IRC08:33
YanXingAngongysh: still coding08:33
gongyshyes08:33
gongyshYanXingAn, don't you want a bp for it?08:33
joxyukiok, what is the big problme?08:33
gongysha spec?08:33
*** cloudrancher has joined #openstack-meeting08:33
gongyshYanXingAn, spec is not required of course.08:34
*** electrofelix has joined #openstack-meeting08:35
*** alexchadin has quit IRC08:35
*** alexchadin has joined #openstack-meeting08:35
*** heyongli has quit IRC08:35
*** heyongli has joined #openstack-meeting08:35
gongysh#topic open discussion08:36
*** openstack changes topic to "open discussion (Meeting topic: tacker)"08:36
gongyshdkushwaha, do you have some thing to talk?08:36
dkushwahagongysh, nguyenhai, I am reusing this patch,  https://review.openstack.org/#/c/48692408:36
patchbotpatch 486924 - tacker - Implement VNF monitoring using Mistral08:36
dkushwahanguyenhai,  But seems vim authentication related issue.08:36
nguyenhaiyes, we have problem with vim authentication08:37
*** alexchad_ has joined #openstack-meeting08:37
gongyshdkushwaha, this is why YanXingAn is doing barbican stuff.08:37
dkushwahaDo you have any idea about that, as i missed it08:37
*** shaohe_feng has quit IRC08:37
dkushwahagongysh, oh i see08:37
*** alexchadin has quit IRC08:37
gongyshYanXingAn will design a service account which can access all vim credentials.08:38
*** shaohe_feng has joined #openstack-meeting08:38
*** alexchadin has joined #openstack-meeting08:38
dkushwahagongysh, is this specific to conductor ?08:38
gongyshyes08:38
gongyshonly conductor needs that.08:39
dkushwahayup08:39
*** psachin` is now known as psachin08:39
*** cloudrancher has quit IRC08:39
dkushwahaYanXingAn, do you have some draft for that?08:39
gongysh#topic berlin summit08:39
*** openstack changes topic to "berlin summit (Meeting topic: tacker)"08:39
gongyshberlin summit is calling for presentation.08:40
*** salv-orlando has quit IRC08:40
gongyshat vancouver, we have sfc, multi-vim talks.08:40
YanXingAndkushwaha: will have a draft before next meeting08:40
dkushwahaYanXingAn, great, thanks08:41
*** kaisers has joined #openstack-meeting08:41
gongyshany tacker oriented presentations are welcome.08:42
*** alexchad_ has quit IRC08:42
gongyshseems we have no other stuff to talk.08:42
gongyshahrechny, look forward to you csar draft design next meeting. thanks08:43
gongysh#endmeeting08:43
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:43
openstackMeeting ended Tue Jun 26 08:43:49 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:43
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-06-26-08.02.html08:43
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-06-26-08.02.txt08:43
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-06-26-08.02.log.html08:43
*** heyongli has quit IRC08:45
*** heyongli has joined #openstack-meeting08:46
*** YanXingAn has quit IRC08:46
*** YanXingAn has joined #openstack-meeting08:47
*** cloudrancher has joined #openstack-meeting08:47
*** shaohe_feng has quit IRC08:47
*** shaohe_feng has joined #openstack-meeting08:48
*** psahoo has joined #openstack-meeting08:48
*** YanXingAn has quit IRC08:49
*** priteau has joined #openstack-meeting08:54
*** phuoc has left #openstack-meeting08:54
*** cloudrancher has quit IRC08:55
*** cloudrancher has joined #openstack-meeting08:55
*** cloudrancher has quit IRC08:55
*** heyongli has quit IRC08:56
*** diman has joined #openstack-meeting08:56
*** heyongli has joined #openstack-meeting08:56
*** iyamahat has joined #openstack-meeting08:56
*** brault has quit IRC08:57
*** shaohe_feng has quit IRC08:58
*** iyamahat has quit IRC08:59
*** shaohe_feng has joined #openstack-meeting09:00
*** jchhatbar is now known as janki09:01
tetsuroo/09:04
*** armax has quit IRC09:06
*** heyongli has quit IRC09:06
*** heyongli has joined #openstack-meeting09:06
*** ricolin has quit IRC09:07
*** shaohe_feng has quit IRC09:08
*** shaohe_feng has joined #openstack-meeting09:09
*** brault has joined #openstack-meeting09:13
*** slaweq has joined #openstack-meeting09:15
*** heyongli has quit IRC09:16
*** heyongli has joined #openstack-meeting09:16
*** lifeless has quit IRC09:16
*** kaisers_ has quit IRC09:18
*** shaohe_feng has quit IRC09:18
*** aloga has joined #openstack-meeting09:19
*** shaohe_feng has joined #openstack-meeting09:20
*** psahoo has quit IRC09:21
*** slaweq has quit IRC09:23
*** heyongli has quit IRC09:26
*** heyongli has joined #openstack-meeting09:27
*** qwebirc77170 has joined #openstack-meeting09:27
*** shaohe_feng has quit IRC09:28
*** shaohe_feng has joined #openstack-meeting09:29
*** psahoo has joined #openstack-meeting09:33
*** wanghao has quit IRC09:36
*** heyongli has quit IRC09:37
*** heyongli has joined #openstack-meeting09:37
*** wanghao has joined #openstack-meeting09:37
*** wanghao has quit IRC09:37
*** wanghao has joined #openstack-meeting09:37
*** wanghao has quit IRC09:38
*** wanghao has joined #openstack-meeting09:38
*** shaohe_feng has quit IRC09:39
*** wanghao has quit IRC09:39
*** wanghao has joined #openstack-meeting09:39
*** akozhevnikov has quit IRC09:40
*** salv-orlando has joined #openstack-meeting09:41
*** shaohe_feng has joined #openstack-meeting09:41
*** salv-orlando has quit IRC09:46
*** strigazi_ is now known as strigazi09:47
*** heyongli has quit IRC09:47
*** heyongli has joined #openstack-meeting09:47
*** shaohe_feng has quit IRC09:49
*** shaohe_feng has joined #openstack-meeting09:50
*** qwebirc77170 has left #openstack-meeting09:55
*** heyongli has quit IRC09:57
*** heyongli has joined #openstack-meeting09:57
*** shaohe_feng has quit IRC09:59
*** shaohe_feng has joined #openstack-meeting10:00
*** Li_Jiale has quit IRC10:01
*** heyongli has quit IRC10:07
*** heyongli has joined #openstack-meeting10:08
*** shaohe_feng has quit IRC10:09
*** shaohe_feng has joined #openstack-meeting10:10
*** ttsiouts has quit IRC10:14
*** jchhatbar has joined #openstack-meeting10:17
*** heyongli has quit IRC10:18
*** heyongli has joined #openstack-meeting10:18
*** julim has joined #openstack-meeting10:18
*** janki has quit IRC10:19
*** shaohe_feng has quit IRC10:20
*** numans has quit IRC10:20
*** shaohe_feng has joined #openstack-meeting10:21
*** numans has joined #openstack-meeting10:25
*** heyongli has quit IRC10:28
*** heyongli has joined #openstack-meeting10:28
*** caobin has quit IRC10:29
*** caobin has joined #openstack-meeting10:29
*** shaohe_feng has quit IRC10:30
*** shaohe_feng has joined #openstack-meeting10:31
*** boden has joined #openstack-meeting10:38
*** heyongli has quit IRC10:38
*** heyongli has joined #openstack-meeting10:38
*** boden has quit IRC10:38
*** joxyuki has quit IRC10:39
*** shaohe_feng has quit IRC10:40
*** salv-orlando has joined #openstack-meeting10:42
*** shaohe_feng has joined #openstack-meeting10:42
*** caobin has quit IRC10:47
*** salv-orlando has quit IRC10:47
*** slaweq has joined #openstack-meeting10:48
*** heyongli has quit IRC10:48
*** heyongli has joined #openstack-meeting10:49
*** shaohe_feng has quit IRC10:50
*** shaohe_feng has joined #openstack-meeting10:51
*** cloudrancher has joined #openstack-meeting10:56
*** yamamoto has quit IRC10:58
*** heyongli has quit IRC10:59
*** heyongli has joined #openstack-meeting10:59
*** andreas_s has quit IRC11:00
*** andreas_s has joined #openstack-meeting11:00
*** andreas_s has quit IRC11:00
*** shaohe_feng has quit IRC11:01
*** andreas_s has joined #openstack-meeting11:01
*** andreas_s has quit IRC11:02
*** shaohe_feng has joined #openstack-meeting11:02
*** andreas_s has joined #openstack-meeting11:02
*** ttsiouts has joined #openstack-meeting11:02
*** andreas_s has quit IRC11:03
*** dkushwaha has quit IRC11:06
*** yamamoto has joined #openstack-meeting11:07
*** heyongli has quit IRC11:09
*** tetsuro has quit IRC11:09
*** heyongli has joined #openstack-meeting11:09
*** shaohe_feng has quit IRC11:11
*** shaohe_feng has joined #openstack-meeting11:14
*** andreas_s has joined #openstack-meeting11:17
*** heyongli has quit IRC11:19
*** heyongli has joined #openstack-meeting11:19
*** shaohe_feng has quit IRC11:21
*** shaohe_feng has joined #openstack-meeting11:22
*** pchavva has joined #openstack-meeting11:22
*** slaweq has quit IRC11:23
*** andreas_s has quit IRC11:26
*** heyongli has quit IRC11:29
*** heyongli has joined #openstack-meeting11:30
*** yamamoto has quit IRC11:30
*** sambetts|afk is now known as sambetts11:31
*** shaohe_feng has quit IRC11:31
*** shaohe_feng has joined #openstack-meeting11:32
*** yamamoto has joined #openstack-meeting11:33
*** andreas_s has joined #openstack-meeting11:35
*** longkb has quit IRC11:36
*** tpsilva has joined #openstack-meeting11:37
*** cloudrancher has quit IRC11:37
*** alexchad_ has joined #openstack-meeting11:38
*** alexchadin has quit IRC11:38
*** heyongli has quit IRC11:40
*** heyongli has joined #openstack-meeting11:40
*** shaohe_feng has quit IRC11:42
*** shaohe_feng has joined #openstack-meeting11:42
*** andreas_s has quit IRC11:44
*** andreas_s has joined #openstack-meeting11:48
*** heyongli has quit IRC11:50
*** heyongli has joined #openstack-meeting11:50
*** shaohe_feng has quit IRC11:52
*** shaohe_feng has joined #openstack-meeting11:53
*** cloudrancher has joined #openstack-meeting11:55
*** rbartal has quit IRC12:00
*** heyongli has quit IRC12:00
*** heyongli has joined #openstack-meeting12:00
*** shaohe_feng has quit IRC12:02
*** rbartal has joined #openstack-meeting12:02
*** shaohe_feng has joined #openstack-meeting12:04
*** julim has quit IRC12:09
*** eharney has quit IRC12:10
*** heyongli has quit IRC12:10
*** heyongli has joined #openstack-meeting12:11
*** raildo has joined #openstack-meeting12:12
*** shaohe_feng has quit IRC12:12
*** shaohe_feng has joined #openstack-meeting12:13
*** bobh has joined #openstack-meeting12:13
*** imcsk8 has quit IRC12:16
*** alexchad_ has quit IRC12:17
*** alexpilotti has joined #openstack-meeting12:18
*** alexpil__ has joined #openstack-meeting12:19
*** heyongli has quit IRC12:21
*** heyongli has joined #openstack-meeting12:21
*** alexpilo_ has quit IRC12:21
*** alexpilotti has quit IRC12:23
*** shaohe_feng has quit IRC12:23
*** imcsk8 has joined #openstack-meeting12:23
*** shaohe_feng has joined #openstack-meeting12:24
*** slaweq has joined #openstack-meeting12:25
*** ttsiouts has quit IRC12:25
*** cloudrancher has quit IRC12:27
*** cloudran_ has joined #openstack-meeting12:27
*** fnaval has joined #openstack-meeting12:28
*** slaweq has quit IRC12:29
*** alexpil__ has quit IRC12:30
*** heyongli has quit IRC12:31
*** alexchadin has joined #openstack-meeting12:31
*** heyongli has joined #openstack-meeting12:31
*** shaohe_feng has quit IRC12:33
*** shaohe_feng has joined #openstack-meeting12:34
*** alexpilotti has joined #openstack-meeting12:39
*** heyongli has quit IRC12:41
*** heyongli has joined #openstack-meeting12:41
*** shaohe_feng has quit IRC12:43
*** alexpilotti has quit IRC12:44
*** shaohe_feng has joined #openstack-meeting12:45
*** andreas_s has quit IRC12:48
*** ttsiouts has joined #openstack-meeting12:49
*** andreas_s has joined #openstack-meeting12:50
*** mriedem has joined #openstack-meeting12:51
*** heyongli has quit IRC12:51
*** heyongli has joined #openstack-meeting12:52
*** shaohe_feng has quit IRC12:53
*** shaohe_feng has joined #openstack-meeting12:54
*** dmacpher has quit IRC12:55
*** andreas_s has quit IRC12:57
*** cloudran_ has quit IRC12:57
*** edmondsw has joined #openstack-meeting12:58
*** cloudrancher has joined #openstack-meeting12:58
*** cloudrancher has quit IRC12:59
*** andreas_s has joined #openstack-meeting12:59
*** yamamoto has quit IRC13:01
*** ian_ott has joined #openstack-meeting13:01
*** r-mibu has joined #openstack-meeting13:01
*** heyongli has quit IRC13:02
*** rossella_s has joined #openstack-meeting13:02
*** heyongli has joined #openstack-meeting13:02
*** aloga has quit IRC13:02
*** shaohe_feng has quit IRC13:04
*** shaohe_feng has joined #openstack-meeting13:04
*** zhhuabj has joined #openstack-meeting13:10
*** alexpilotti has joined #openstack-meeting13:11
*** heyongli has quit IRC13:12
*** heyongli has joined #openstack-meeting13:12
*** julim has joined #openstack-meeting13:13
*** mjturek has joined #openstack-meeting13:14
*** shaohe_feng has quit IRC13:14
*** shaohe_feng has joined #openstack-meeting13:15
*** dustins has joined #openstack-meeting13:17
*** alexpilo_ has joined #openstack-meeting13:20
*** heyongli has quit IRC13:22
*** heyongli has joined #openstack-meeting13:22
*** eharney has joined #openstack-meeting13:23
*** alexpilotti has quit IRC13:24
*** shaohe_feng has quit IRC13:24
*** shaohe_feng has joined #openstack-meeting13:25
*** frickler has quit IRC13:26
*** frickler has joined #openstack-meeting13:26
*** bobh has quit IRC13:29
*** yamamoto has joined #openstack-meeting13:29
*** alexpilotti has joined #openstack-meeting13:30
*** heyongli has quit IRC13:32
*** alexchadin has quit IRC13:33
*** heyongli has joined #openstack-meeting13:33
*** fabian_ has joined #openstack-meeting13:33
*** alexpilo_ has quit IRC13:33
*** shaohe_feng has quit IRC13:34
*** shaohe_feng has joined #openstack-meeting13:35
*** awaugama has joined #openstack-meeting13:36
*** e0ne has quit IRC13:39
*** alexpilo_ has joined #openstack-meeting13:40
*** hongbin has joined #openstack-meeting13:42
*** heyongli has quit IRC13:43
*** heyongli has joined #openstack-meeting13:43
*** alexpilotti has quit IRC13:43
*** e0ne has joined #openstack-meeting13:44
*** shaohe_feng has quit IRC13:45
*** shaohe_feng has joined #openstack-meeting13:47
*** alexpilotti has joined #openstack-meeting13:50
*** alexchadin has joined #openstack-meeting13:51
*** heyongli has quit IRC13:53
*** heyongli has joined #openstack-meeting13:53
*** alexpilo_ has quit IRC13:53
*** shaohe_feng has quit IRC13:55
*** shaohe_feng has joined #openstack-meeting13:56
*** alexpilo_ has joined #openstack-meeting14:00
*** julim_ has joined #openstack-meeting14:01
*** jchhatbar is now known as janki14:01
*** julim_ has quit IRC14:02
*** njohnston has joined #openstack-meeting14:02
*** rfolco_ is now known as rfolco14:03
*** dkushwaha has joined #openstack-meeting14:03
*** heyongli has quit IRC14:03
*** julim_ has joined #openstack-meeting14:03
*** alexpilotti has quit IRC14:03
*** heyongli has joined #openstack-meeting14:03
*** julim has quit IRC14:04
*** rpioso|afk is now known as rpioso14:04
*** shaohe_feng has quit IRC14:05
*** rossella_s has quit IRC14:07
*** shaohe_feng has joined #openstack-meeting14:08
*** dkushwaha has quit IRC14:08
*** alexchadin has quit IRC14:08
*** rossella_s has joined #openstack-meeting14:09
*** psahoo has quit IRC14:09
*** artom has joined #openstack-meeting14:09
*** alexpilotti has joined #openstack-meeting14:10
*** jamesmcarthur_ has joined #openstack-meeting14:12
*** heyongli has quit IRC14:13
*** rossella_s has quit IRC14:13
*** alexpilo_ has quit IRC14:13
*** heyongli has joined #openstack-meeting14:14
*** shaohe_feng has quit IRC14:15
*** wxy| has joined #openstack-meeting14:16
*** shaohe_feng has joined #openstack-meeting14:16
*** armax has joined #openstack-meeting14:16
*** cdub has joined #openstack-meeting14:17
*** fabian_ has quit IRC14:17
*** slaweq has joined #openstack-meeting14:17
*** rossella_s has joined #openstack-meeting14:17
*** heyongli has quit IRC14:18
*** shaohe_feng has quit IRC14:18
*** heyongli has joined #openstack-meeting14:19
*** shaohe_feng has joined #openstack-meeting14:19
*** alexpilo_ has joined #openstack-meeting14:20
*** cdub has quit IRC14:22
*** e0ne has quit IRC14:23
*** alexpilotti has quit IRC14:23
*** heyongli has quit IRC14:24
*** heyongli has joined #openstack-meeting14:24
*** eharney has quit IRC14:25
*** shintaro has joined #openstack-meeting14:25
*** e0ne has joined #openstack-meeting14:25
*** shaohe_feng has quit IRC14:26
*** felipemonteiro has joined #openstack-meeting14:26
*** shaohe_feng has joined #openstack-meeting14:26
*** psachin has quit IRC14:28
*** felipemonteiro_ has joined #openstack-meeting14:29
*** alexpilotti has joined #openstack-meeting14:30
*** felipemonteiro has quit IRC14:33
*** alexpilo_ has quit IRC14:33
*** heyongli has quit IRC14:34
*** heyongli has joined #openstack-meeting14:34
*** shaohe_feng has quit IRC14:36
*** shaohe_feng has joined #openstack-meeting14:36
*** efried has quit IRC14:40
*** antosh has joined #openstack-meeting14:40
*** alexpilo_ has joined #openstack-meeting14:40
*** efried has joined #openstack-meeting14:40
*** bobh has joined #openstack-meeting14:40
*** heyongli has quit IRC14:40
*** shaohe_feng has quit IRC14:40
*** eharney has joined #openstack-meeting14:42
*** alexpilotti has quit IRC14:43
*** rossella_s has quit IRC14:49
*** alexpilotti has joined #openstack-meeting14:50
*** rossella_s has joined #openstack-meeting14:51
*** gongysh has quit IRC14:53
*** alexpilo_ has quit IRC14:53
*** cloudrancher has joined #openstack-meeting14:55
*** shintaro has quit IRC14:57
*** alexpilo_ has joined #openstack-meeting15:00
*** alexpilotti has quit IRC15:03
*** kaisers_ has joined #openstack-meeting15:05
*** hemna_ has joined #openstack-meeting15:06
*** e0ne has quit IRC15:07
*** markstur has joined #openstack-meeting15:09
*** alexpilotti has joined #openstack-meeting15:10
*** e0ne has joined #openstack-meeting15:11
*** jamesmca_ has joined #openstack-meeting15:12
*** janki has quit IRC15:13
*** alexpilo_ has quit IRC15:13
*** munimeha2 has joined #openstack-meeting15:17
*** felipemonteiro_ has quit IRC15:19
*** felipemonteiro_ has joined #openstack-meeting15:19
*** cloudrancher has quit IRC15:20
*** alexpilo_ has joined #openstack-meeting15:20
*** cloudrancher has joined #openstack-meeting15:21
*** felipemonteiro has joined #openstack-meeting15:22
*** alexpilotti has quit IRC15:24
*** felipemonteiro_ has quit IRC15:24
*** Guest1606 has quit IRC15:28
*** alexpilotti has joined #openstack-meeting15:30
*** andreas_s has quit IRC15:32
*** heyongli has joined #openstack-meeting15:32
*** felipemonteiro has quit IRC15:32
*** felipemonteiro has joined #openstack-meeting15:32
*** felipemonteiro_ has joined #openstack-meeting15:33
*** alexpilo_ has quit IRC15:33
*** shaohe_feng has joined #openstack-meeting15:34
*** heyongli has quit IRC15:34
*** heyongli has joined #openstack-meeting15:34
*** shaohe_feng has quit IRC15:36
*** ttsiouts has quit IRC15:36
*** felipemonteiro has quit IRC15:37
*** shaohe_feng has joined #openstack-meeting15:37
*** eharney has quit IRC15:39
*** alexpilo_ has joined #openstack-meeting15:40
*** ricolin has joined #openstack-meeting15:42
*** pcaruana has quit IRC15:42
*** ttsiouts has joined #openstack-meeting15:43
*** alexpilotti has quit IRC15:44
*** heyongli has quit IRC15:44
*** heyongli has joined #openstack-meeting15:45
*** shaohe_feng has quit IRC15:47
*** shaohe_feng has joined #openstack-meeting15:47
*** gyee has joined #openstack-meeting15:50
*** alexpilotti has joined #openstack-meeting15:50
*** dklyle has quit IRC15:50
*** chyka has joined #openstack-meeting15:52
*** dklyle has joined #openstack-meeting15:53
*** cloudrancher has quit IRC15:53
*** alexpilo_ has quit IRC15:54
*** cloudrancher has joined #openstack-meeting15:54
*** heyongli has quit IRC15:55
*** heyongli has joined #openstack-meeting15:55
*** shaohe_feng has quit IRC15:57
*** shaohe_feng has joined #openstack-meeting16:00
*** mlavalle has joined #openstack-meeting16:00
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Jun 26 16:00:22 2018 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
slaweqhi16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
mlavalleo/16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
*** alexpilo_ has joined #openstack-meeting16:00
njohnstono/16:00
slaweqhaleyb: are You around?16:01
* slaweq is on airport so sorry if there will be some issues with connection16:02
slaweqok, I maybe haleyb will join later16:02
slaweqlet's start16:02
mlavallegoing on vaction?16:02
slaweqno, I'm going back home from internal training16:02
mlavalleahhh16:03
mlavallehave a safe flight16:03
slaweqthx :)16:03
slaweq#topic Actions from previous meetings16:03
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:03
slaweqwe have only one action from last week16:03
slaweqnjohnston to look into adding Grafana dasboard for stable branches16:03
*** gyee has quit IRC16:03
*** alexpilotti has quit IRC16:03
njohnstonI have started working on that but got pulled into some other things; I plan to push something for people to review later this week16:04
njohnstonApologies for the delay16:04
slaweqok, thx for update16:04
slaweqno problem, it's not very urgent :)16:04
slaweq#action njohnston to look into adding Grafana dasboard for stable branches16:04
slaweqYou have it for next week :)16:04
mlavalleno apologies needed. we all have sponsors who help us to pay the bills16:05
njohnstonthanks16:05
mlavalleand they have their priorities16:05
slaweqmlavalle: well said :)16:05
*** heyongli has quit IRC16:05
slaweq#topic Grafana16:05
*** heyongli has joined #openstack-meeting16:05
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:05
slaweqhttp://grafana.openstack.org/dashboard/db/neutron-failure-rate16:05
slaweqgenerally it doesn't looks very bad this week IMO16:07
*** shaohe_feng has quit IRC16:07
slaweqbut I don't know why there is some "gap" with graphs yesterday and on Sunday16:07
mlavalleThe tempest linux bridge job has had some failures16:07
slaweqmlavalle: are You talking about this "spike" in gate queue?16:08
mlavalleyeah16:08
slaweqI was checking that and I found few issues not related to neutron, like e.g. 503 errors from cinder: http://logs.openstack.org/63/323963/61/check/neutron-tempest-linuxbridge/1d56164/logs/testr_results.html.gz16:08
mlavalleyes, I agree, What I've seens is someting about Nova tagging testing16:09
slaweqand I think that this high failure rate is also because there was no many runs counted this time16:09
*** AndreyS has quit IRC16:09
*** shaohe_feng has joined #openstack-meeting16:09
*** e0ne has quit IRC16:09
slaweqas I went yesterday through some of failed jobs and there wasn't many failed examples of this job16:10
mlavalleok, let's keep an eye on it and see what happens over the next few days16:10
slaweqabout tagging test failure I have it for one of next topics :)16:10
*** diman has quit IRC16:10
*** alexpilotti has joined #openstack-meeting16:10
slaweqmlavalle: I agree16:10
*** rossella_s has quit IRC16:10
mlavallemaybe I am just pissed off with this job because it has failed in some of my patches ;-)16:11
slaweqso let's now talk about some specific jobs16:11
slaweqmlavalle: LOL16:11
slaweqmaybe16:11
slaweq#topic Unit tests16:11
*** openstack changes topic to "Unit tests (Meeting topic: neutron_ci)"16:11
slaweqI added UT as a topic today because I found few failures with some timeout16:11
slaweqlike e.g.: http://logs.openstack.org/58/565358/14/check/openstack-tox-py35/aa30b12/job-output.txt.gz or http://logs.openstack.org/03/563803/9/check/openstack-tox-py35/a50de4a/job-output.txt.gz16:11
slaweqand it was always py3516:12
*** cloudrancher has quit IRC16:12
*** cdub has joined #openstack-meeting16:12
slaweqor maybe You saw something similar with py27 but I didn't spotted it yet16:12
*** harlowja has joined #openstack-meeting16:12
mlavallemhhh, let me see16:12
slaweqI think that this should be investigated16:12
*** e0ne has joined #openstack-meeting16:13
mlavalleno, I saw I timeout a few minutes ago, but it wasn't unit tests16:13
*** alexpilo_ has quit IRC16:13
*** r-mibu has quit IRC16:14
*** rossella_s has joined #openstack-meeting16:14
njohnstonI have seen tempest get killed due to timeouts as well, for example: http://logs.openstack.org/61/566961/4/check/neutron-tempest-iptables_hybrid/c70896b/job-output.txt.gz#_2018-06-26_08_55_07_63863116:14
*** ahrechny has quit IRC16:14
*** heyongli has quit IRC16:15
slaweqI think that we should report this one as a bug, check how long "good" run takes and then check what issue it is maybe16:15
*** heyongli has joined #openstack-meeting16:15
mlavallesounds like a plan16:16
slaweqand I will try to investigate this UT tests16:16
slaweq#actions slaweq will report and investigate py35 timeouts16:16
slaweqnjohnston: tempest is probably different issue16:16
slaweqI know that it happens sometimes and we probably also should check16:17
*** shaohe_feng has quit IRC16:17
slaweqnjohnston: would You like to investigate it?16:18
slaweqok, I will report it as a bug also and I will see if I will be able to check it this week16:19
*** shaohe_feng has joined #openstack-meeting16:19
slaweq#action slaweq reports a bug with tempest timeouts, like on http://logs.openstack.org/61/566961/4/check/neutron-tempest-iptables_hybrid/c70896b/job-output.txt.gz16:20
slaweqok, let's go to next one16:20
slaweq#topic Functional16:20
*** openstack changes topic to "Functional (Meeting topic: neutron_ci)"16:20
*** alexpilo_ has joined #openstack-meeting16:20
slaweqI saw at least few times something like:     * http://logs.openstack.org/58/574058/10/check/neutron-functional/f38d685/logs/testr_results.html.gz16:20
slaweqit might be related to  https://bugs.launchpad.net/neutron/+bug/168702716:21
openstackLaunchpad bug 1687027 in neutron "test_walk_versions tests fail with "IndexError: tuple index out of range" after timeout" [High,Confirmed]16:21
slaweqit happens at least few times this week so I think we should also dig into it and maybe find a way how to fix it :)16:21
*** alexpilotti has quit IRC16:23
slaweqand those are different tests failing but with similar error, second example from last few days:     * http://logs.openstack.org/61/567461/4/check/neutron-functional/81d69a4/logs/testr_results.html.gz16:24
slaweqany volunteer to debug this issue?16:24
mlavalleo/16:24
slaweqthx mlavalle :)16:24
mlavalleI assigned it to me16:25
slaweq#action mlavalle will check functional tests issue, https://bugs.launchpad.net/neutron/+bug/168702716:25
openstackLaunchpad bug 1687027 in neutron "test_walk_versions tests fail with "IndexError: tuple index out of range" after timeout" [High,Confirmed] - Assigned to Miguel Lavalle (minsel)16:25
slaweqok, thx a lot16:25
slaweqI will not talk about fullstack tests today as it is on quite good failure rate recently and I didn't have time to dig into this one often failing test yet16:25
*** heyongli has quit IRC16:25
slaweqso next topic will be now16:25
*** yangyape_ has joined #openstack-meeting16:26
slaweq#topic Scenarios16:26
*** openstack changes topic to "Scenarios (Meeting topic: neutron_ci)"16:26
*** heyongli has joined #openstack-meeting16:26
slaweqAs mlavalle mentioned before, there is some issue with tagging test from tempest16:26
slaweqbug report: https://bugs.launchpad.net/tempest/+bug/177594716:26
openstackLaunchpad bug 1775947 in tempest "tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest failing" [Medium,Confirmed] - Assigned to Deepak Mourya (mourya007)16:26
mlavalleyeah I've seen it more in api tests16:26
slaweqexample of failure is e.g. on http://logs.openstack.org/44/575444/4/gate/neutron-tempest-linuxbridge/118cc97/logs/testr_results.html.gz16:27
mlavallenot as much in scenrio tests16:27
slaweqahh, yes16:27
slaweqsorry, my bad16:27
slaweqI mixed them as it's "tempest" in both cases :)16:27
slaweqit's in tempest api tests16:27
slaweqso let's change topic to16:27
mlavalleand ye, I see it most;y in the linuxbridge job16:27
*** yangyapeng has quit IRC16:27
slaweq#topic tempest tests16:27
*** openstack changes topic to "tempest tests (Meeting topic: neutron_ci)"16:27
slaweq:)16:27
*** shaohe_feng has quit IRC16:28
slaweqI didn't check if it's only related to linuxbridge job16:28
*** shaohe_feng has joined #openstack-meeting16:28
slaweqwhat I saw often was something like "process exited with errcode 137" on VM16:28
*** jamesmca_ has quit IRC16:28
slaweqduring doing "curl" to metadata service16:28
mlavalleright, I've also seen the message about the metadata service16:29
*** jamesmca_ has joined #openstack-meeting16:29
*** alexpilotti has joined #openstack-meeting16:30
slaweqfor now it is reported against tempest as I suspected at first glance that it might be issue with "too fast" removing of port from instance and doing curl from VM to metadata API16:31
slaweqbut now I'm not sure if that is (only) issue16:31
slaweqI will try to investigate it more as I was already checking it a bit16:31
slaweqbut I'm not sure if I will be able to do it this week16:32
*** cloudrancher has joined #openstack-meeting16:32
slaweq#action slaweq will investigate tagging tempest test issue16:32
*** cdub has quit IRC16:33
*** ricolin has quit IRC16:33
slaweqfrom other issues which I found and wanted to raise here is failure in neutron-tempest-dvr test:16:33
slaweqhttp://logs.openstack.org/14/529814/16/check/neutron-tempest-dvr/d9696d5/logs/testr_results.html.gz16:33
slaweqI think I spotted it only once but maybe You saw it also somewhere?16:33
*** alexpilo_ has quit IRC16:33
*** jamesmca_ has quit IRC16:34
mlavalleNO I haven't seen it16:34
mlavalleBut I'll keep an eye open for it16:34
slaweqthx mlavalle16:35
*** e0ne has quit IRC16:35
*** heyongli has quit IRC16:36
slaweqI also found timeout issue with tempest-py3 test: http://logs.openstack.org/61/566961/4/gate/tempest-full-py3/4ca37f8/job-output.txt.gz16:36
*** heyongli has joined #openstack-meeting16:36
*** cloudrancher has quit IRC16:36
slaweqbut I think that's the same as pointed by njohnston earlier16:36
slaweqso it's already assigned to me :)16:36
mlavallelol16:37
slaweqanything else to add here?16:37
mlavallenot from me16:37
slaweqok, moving to next topic then16:37
slaweq#topic Rally16:37
*** openstack changes topic to "Rally (Meeting topic: neutron_ci)"16:37
slaweqThere was an issue related to rally last week (spike on grafana) but was fixed quickly on rally side after talk with andreykurilin16:37
slaweqit's just FYI :)16:37
mlavalleack16:38
slaweqtoday boden spotted some new issue with rally for stable/queens branch16:38
slaweqhttps://bugs.launchpad.net/neutron/+bug/177871416:38
openstackLaunchpad bug 1778714 in neutron "neutron-rally-neutron fails with `NeutronTrunks.create_and_list_trunk_subports` in in any platform" [Critical,New]16:38
*** shaohe_feng has quit IRC16:38
slaweqbut it looks that patch which he proposed fixes this problem16:38
mlavalleis that a neutron patch?16:38
slaweqI also talked with andreykurilin on rally channel and he confirmed me just before meeting that this should help :)16:38
slaweqyes: https://review.openstack.org/#/c/578104/16:39
patchbotpatch 578104 - neutron (stable/queens) - Use rally 0.12.1 release for stable/pike branch.16:39
*** janki has joined #openstack-meeting16:39
mlavalleok, added to my pile16:39
*** qwebirc17612 has joined #openstack-meeting16:40
slaweqthx mlavalle16:40
*** tesseract has quit IRC16:40
*** alexpilo_ has joined #openstack-meeting16:40
slaweqso I guess we can go to next topic now16:40
slaweq#topic Grenade16:40
*** openstack changes topic to "Grenade (Meeting topic: neutron_ci)"16:40
slaweqI also found issue in neutron-grenade-dvr-multinode job:16:40
slaweqhttp://logs.openstack.org/03/563803/9/check/neutron-grenade-dvr-multinode/13338d9/logs/testr_results.html.gz16:41
slaweqdid You saw it already?16:41
*** shaohe_feng has joined #openstack-meeting16:41
mlavalleI think I did, yeah16:41
slaweqso it't also something which we should check16:41
mlavalleThis specific failure is the instance failing to become active16:42
slaweqit looks for me like something what can be potentially related to neutron16:42
slaweqL2 agent could not configure port or server didn't send notification to nova16:42
slaweqor it's maybe just misconfiguration problem as it stays in error message that it didn't become active in 196 seconds16:43
slaweqand IIRC nova-compute waits for port to be active for 300 seconds by default16:43
*** alexpilotti has quit IRC16:44
*** heyongli has quit IRC16:46
*** heyongli has joined #openstack-meeting16:46
slaweqstrange thing for me is that this instance uuid is not in nova-compute logs at all16:47
slaweqbut I don't know exactly if it should be there16:47
*** qwebirc17612 has quit IRC16:47
*** njohnston has quit IRC16:48
*** shaohe_feng has quit IRC16:48
slaweqmlavalle: but in neutron-server logs there are some errors relted to dvr somehow: http://logs.openstack.org/03/563803/9/check/neutron-grenade-dvr-multinode/13338d9/logs/screen-q-svc.txt.gz?level=ERROR16:48
*** njohnston has joined #openstack-meeting16:48
*** rossella_s has quit IRC16:48
slaweqcheck first 3 lines16:48
slaweqdo You think that it might be related?16:49
*** lpetrut has quit IRC16:50
*** alexpilotti has joined #openstack-meeting16:50
*** shaohe_feng has joined #openstack-meeting16:50
slaweqmlavalle: are You here still? :)16:51
mlavallemhhh16:51
mlavalleI was looking16:51
slaweqahh, ok :)16:51
mlavalleat first glance I don't think so16:51
mlavallethe messages refer to DVR ports16:51
mlavalleand multiple port bindings only apply to compute ports16:52
mlavallebut I will keep an eye open anyways16:52
mlavallethanks for bringingi  up16:52
slaweqthx16:52
slaweqok, that's all from me according to failures in CI for this week16:53
slaweqdo You have anything else?16:53
mlavallenope, thanks flor the thorough update, as usual16:53
*** rossella_s has joined #openstack-meeting16:53
njohnstonQuick question, do the grafana graphs include just jobs that end in ERROR, or do they also include the increasing number of jobs that end in TIMED_OUT status as reported back in Zuul?16:53
*** alexpilo_ has quit IRC16:53
slaweqnjohnston: good question16:53
slaweqI don't know to be honest16:54
mlavalledon't know either16:54
mlavallebetter ask in the infra channel16:54
slaweqaccording to config file of dashboard: https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/neutron.yaml16:55
slaweqit's FAILURE but I have no idea how this value is calculated :)16:55
njohnstonright16:55
njohnstonOK, i will ask in infra channel and report back16:55
slaweqnjohnston: thx a lot16:55
slaweqif You don't have anything else I have one more thing to tell You16:56
njohnston#action njohnston to ask infra team if TIMED_OUT is included in FAILURE for grafana graphs16:56
njohnstongo ahead16:56
*** heyongli has quit IRC16:56
slaweqnext 2 weeks I will be on PTO (\o/) so I will not be able to do this meeting16:56
*** heyongli has joined #openstack-meeting16:56
slaweqYou will need to find someone else who will replace me or just cancel them16:57
*** yamahata has joined #openstack-meeting16:57
slaweqI will be able to chair it again on 17.0716:57
mlavalleNext week will be slow anyway with the July 4th holiday anywas16:57
mlavalleso probably it will be best to cancel it16:57
mlavallebut I can run it on the 10th16:57
slaweqok, I will send email about canceling next week's meeting16:58
slaweqand thx for handling 10th of July :)16:58
mlavalleif you trust I can handle it, that is16:58
njohnstonthanks mlavalle16:58
slaweq#action slaweq will cancel next week's meeting16:58
*** shaohe_feng has quit IRC16:58
slaweqmlavalle: You will do it better than me for sure :)16:58
mlavalledoubt it. I'll do it anyway16:59
slaweqok, thx for attending today16:59
* haleyb waves in the final minute, just got back and will go through the logs16:59
slaweqhi haleyb :)16:59
slaweqsee You all16:59
haleybbye slaweq :)16:59
mlavallewhat airport are you at slaweq ?16:59
slaweq#endmeeting16:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:59
openstackMeeting ended Tue Jun 26 16:59:41 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-06-26-16.00.html16:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-06-26-16.00.txt16:59
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-06-26-16.00.log.html16:59
slaweqhaleyb: bye :)16:59
slaweqmlavalle: Munich16:59
mlavallehave a safe flight!17:00
slaweqthanks17:00
*** alexpilo_ has joined #openstack-meeting17:00
*** shaohe_feng has joined #openstack-meeting17:01
*** alexpilotti has quit IRC17:03
*** sambetts is now known as sambetts|afk17:04
*** wxy| has quit IRC17:04
*** heyongli has quit IRC17:06
*** heyongli has joined #openstack-meeting17:07
*** shaohe_feng has quit IRC17:09
*** shaohe_feng has joined #openstack-meeting17:10
*** harlowja has quit IRC17:10
*** alexpilotti has joined #openstack-meeting17:10
*** e0ne has joined #openstack-meeting17:11
*** yamamoto has quit IRC17:12
*** alexpilo_ has quit IRC17:13
*** felipemonteiro_ has quit IRC17:14
*** ttsiouts has quit IRC17:14
*** felipemonteiro_ has joined #openstack-meeting17:14
*** yamamoto has joined #openstack-meeting17:14
*** ttsiouts has joined #openstack-meeting17:15
*** heyongli has quit IRC17:17
*** heyongli has joined #openstack-meeting17:17
*** e0ne has quit IRC17:18
*** shaohe_feng has quit IRC17:19
*** gyee has joined #openstack-meeting17:20
*** ttsiouts has quit IRC17:20
*** yamamoto has quit IRC17:20
*** shaohe_feng has joined #openstack-meeting17:20
*** alexpilo_ has joined #openstack-meeting17:20
*** alexpilotti has quit IRC17:23
*** heyongli has quit IRC17:27
*** heyongli has joined #openstack-meeting17:27
*** shaohe_feng has quit IRC17:29
*** alexpilotti has joined #openstack-meeting17:30
*** shaohe_feng has joined #openstack-meeting17:30
*** alexpilo_ has quit IRC17:34
*** heyongli has quit IRC17:37
*** heyongli has joined #openstack-meeting17:37
*** Swami has joined #openstack-meeting17:39
*** Swami_ has joined #openstack-meeting17:39
*** shaohe_feng has quit IRC17:39
*** slaweq has quit IRC17:40
*** alexpilo_ has joined #openstack-meeting17:40
*** shaohe_feng has joined #openstack-meeting17:42
*** tssurya has quit IRC17:43
*** alexpilotti has quit IRC17:43
*** felipemonteiro__ has joined #openstack-meeting17:44
*** kopecmartin has quit IRC17:46
*** heyongli has quit IRC17:47
*** cdub has joined #openstack-meeting17:47
*** heyongli has joined #openstack-meeting17:48
*** felipemonteiro_ has quit IRC17:48
*** andreas_s has joined #openstack-meeting17:48
*** felipemonteiro__ has quit IRC17:49
*** felipemonteiro__ has joined #openstack-meeting17:49
*** shaohe_feng has quit IRC17:50
*** alexpilotti has joined #openstack-meeting17:50
*** shaohe_feng has joined #openstack-meeting17:51
*** electrofelix has quit IRC17:52
*** andreas_s has quit IRC17:53
*** felipemonteiro has joined #openstack-meeting17:53
*** felipemonteiro__ has quit IRC17:54
*** alexpilo_ has quit IRC17:54
*** heyongli has quit IRC17:58
*** heyongli has joined #openstack-meeting17:58
*** rossella_s has quit IRC17:58
*** shaohe_feng has quit IRC18:00
*** alexpilo_ has joined #openstack-meeting18:00
*** felipemonteiro has quit IRC18:01
*** shaohe_feng has joined #openstack-meeting18:01
*** felipemonteiro has joined #openstack-meeting18:01
*** rossella_s has joined #openstack-meeting18:02
*** gary_perkins has quit IRC18:03
*** armstrong has joined #openstack-meeting18:03
*** alexpilotti has quit IRC18:03
*** felipemonteiro_ has joined #openstack-meeting18:08
*** heyongli has quit IRC18:08
*** gary_perkins has joined #openstack-meeting18:08
*** heyongli has joined #openstack-meeting18:08
*** shaohe_feng has quit IRC18:10
*** alexpilotti has joined #openstack-meeting18:10
*** felipemonteiro has quit IRC18:11
*** shaohe_feng has joined #openstack-meeting18:11
*** cdub has quit IRC18:14
*** alexpilo_ has quit IRC18:14
*** armstrong has quit IRC18:14
*** armstrong has joined #openstack-meeting18:15
*** eharney has joined #openstack-meeting18:16
*** yamamoto has joined #openstack-meeting18:16
*** heyongli has quit IRC18:18
*** heyongli has joined #openstack-meeting18:19
*** cdub has joined #openstack-meeting18:19
*** armstrong has quit IRC18:19
*** alexpilo_ has joined #openstack-meeting18:20
*** yamamoto has quit IRC18:20
*** shaohe_feng has quit IRC18:20
*** yamamoto has joined #openstack-meeting18:21
*** shaohe_feng has joined #openstack-meeting18:21
*** alexpilotti has quit IRC18:23
*** ekcs has joined #openstack-meeting18:26
*** rossella_s has quit IRC18:28
*** heyongli has quit IRC18:28
*** heyongli has joined #openstack-meeting18:29
*** alexpilotti has joined #openstack-meeting18:30
*** shaohe_feng has quit IRC18:31
*** dmellado has quit IRC18:32
*** gouthamr has quit IRC18:32
*** rossella_s has joined #openstack-meeting18:33
*** antosh has quit IRC18:33
*** stevebaker has quit IRC18:33
*** alexpilo_ has quit IRC18:34
*** slaweq has joined #openstack-meeting18:34
*** shaohe_feng has joined #openstack-meeting18:35
*** heyongli has quit IRC18:39
*** heyongli has joined #openstack-meeting18:39
*** alexpilo_ has joined #openstack-meeting18:40
*** shaohe_feng has quit IRC18:41
*** shaohe_feng has joined #openstack-meeting18:43
*** alexpilotti has quit IRC18:44
*** heyongli has quit IRC18:49
*** heyongli has joined #openstack-meeting18:49
*** alexpilotti has joined #openstack-meeting18:50
*** antosh has joined #openstack-meeting18:50
*** shaohe_feng has quit IRC18:51
*** shaohe_feng has joined #openstack-meeting18:52
*** sridharg has quit IRC18:52
*** alexpilo_ has quit IRC18:53
clarkbanyone here for the Infra meeting? we'll get started in a couple minutes18:59
*** heyongli has quit IRC18:59
*** heyongli has joined #openstack-meeting18:59
corvuso/19:00
*** alexpilo_ has joined #openstack-meeting19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue Jun 26 19:01:08 2018 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting19:01
ianwo/19:01
*** jamesmcarthur_ has quit IRC19:01
*** shaohe_feng has quit IRC19:01
clarkbThe agenda hasn't been updated for today, as I'm somewhat underprepared chasing a bunch of stuff this morning as well as watching WC. That said its not too terribly wrong19:02
clarkb#topic Announcements19:02
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:02
*** shaohe_feng has joined #openstack-meeting19:02
clarkbAs mentioned world cup is happening. Big game right now :)19:02
clarkbOther than that don't forget to take the openstack user survey if you run or use an openstack cloud19:03
clarkbprovides valuable feedback to openstack on what is important and what can be improved19:03
* mordred can't take the survey - too much football on tv19:03
*** alexpilotti has quit IRC19:03
*** janki has quit IRC19:04
clarkb#topic Specs Approval19:04
*** openstack changes topic to "Specs Approval (Meeting topic: infra)"19:04
clarkbMonty's spec for future config mgmt is up and no longer WIP. I don't think it is ready for approval but we should all review that this week if we can find time19:04
clarkb#link https://review.openstack.org/#/c/565550/ config mgmt and containers19:04
patchbotpatch 565550 - openstack-infra/infra-specs - Update config management for the Infra Control Plane19:04
fungithanks mordred!19:05
corvusnotmyname: ^ can you clear out patchbot please?19:05
*** Shrews has joined #openstack-meeting19:05
clarkbIf possible I'd like for next week's meeting to be able to have a discussion about it in detail if necessary and we can work to merging that spec the week after (if not next week if everyone agrees with what monty has already written)19:05
mordred\o/19:05
clarkbtl;dr please review :)19:05
*** patchbot has left #openstack-meeting19:06
clarkbthough fungi and I will be travelingthe week after19:06
mordredI tried to list all the things - but to leave some bits open for impl - definitely feedback welcome if I missed something or am just dumb19:06
fungiyup19:06
*** cloudrancher has joined #openstack-meeting19:07
* fungi is disappearing a lot next month19:07
clarkbas for working on this at the PTG I've heard rumor we'll hvae ~3 days of room space as well as aday or two of general help room19:08
* mordred throws paint on fungi to try to defeat the invisibility19:08
clarkbI think that will work well for digging into this in denver19:08
mordredyah. if people are happy with the direction - there are several tasks I think I should be able to knock out by denver19:08
corvusi don't expect us to have any dedicated zuul time at this ptg, so those of us with multiple hats should be able to focus on this too19:08
clarkbI should get an etherpad for the PTG going so that we can start coordinating what happens there vs what happens prior19:09
* clarkb makes a note to get that going today19:09
clarkbGOL! (sorry)19:09
*** heyongli has quit IRC19:09
funginote that there will, again, be a cross-project "helproom" for a couple days where we can still help people with questions about zuul job configuration and the like19:10
corvus(though getting this done unlocks a bunch of neato zuul stuff, like running zuul from containers and more cd of zuul)19:10
corvusfungi: ++19:10
mordredclarkb: I think we're watching different games19:10
*** heyongli has joined #openstack-meeting19:10
*** alexpilotti has joined #openstack-meeting19:10
fungithe futbol is multi-threaded19:10
clarkbmordred: possibly, I was leaving that info out for people that might be avoiding spoilers19:11
clarkbcorvus: yup I think this opens a few exciting followup threads once we get things rolling19:11
fungihighly parallel gol processing19:11
*** cdub has quit IRC19:11
clarkb#topic Priority Efforts19:11
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:11
*** cloudrancher has quit IRC19:12
clarkbExciting updates have happend on the storyboard database query side of things19:12
*** shaohe_feng has quit IRC19:12
clarkbyou might notice that storyboard boards are much quicker now thanks to dhellmann19:12
fungimostly thanks to dhellmann19:12
fungithanks again dhellmann!19:12
fungiit's amazing how much adding an index on a column can speed stuff up ;)19:12
dhellmannknowing we were missing an index made that pretty easy to figure out19:12
* mordred hands dhellmann a fluffy bunny rabbit19:13
* dhellmann gets out the stew pot19:13
mordredmmm. rabbit stew19:13
dhellmannmmm19:13
clarkbfungi: any other storyboard related items worth bringing up?19:13
fungiwe're still struggling a bit on the name-based project urls... something's not quite right with decoding url-escaped slashes in project names19:13
clarkbfungi: does it work without apache in front of it? apache likes to mangle those19:14
fungisee discussion in #storyboard for current status19:14
*** alexpilo_ has quit IRC19:14
fungiyeah, the pecan-based dev server works fine19:14
fungibut when running the api server through apache it's a problem19:14
*** felipemonteiro_ has quit IRC19:15
* mordred is very excited about the future of name-based urls19:15
fungii spent a good chunk of last night experimenting but couldn't come up with a workaround, though i also don't fully grasp the api routing in sb19:15
*** slaweq has quit IRC19:15
fungiso adding debugging wasn't easy19:16
*** shaohe_feng has joined #openstack-meeting19:16
fungiin other news, vitrage migrated all their deliverables from lp to sb on friday and that seems to be going well19:16
fungioh, also important, ianw noticed that the occasional rash of 500 internal server error from write operations may be related to rabbitmq disconnects causing the socket to it to get blocked19:17
clarkbfungi: rabbitmq runs on the same instance though right? odd for there to be disconnects19:17
ianwat first glance at the code, it looked liked it was trying to handle that19:18
fungiyeah, unless it gets restarted or something i suppose19:18
ianwbut something in pika seemed to get itself stuck, if i had to guess19:18
* ianw pats myself on the back for such an excellent, helpful bug report :)19:19
*** felipemonteiro has joined #openstack-meeting19:19
fungiit's more detail than we've gathered on the situation to date19:19
fungiso thanks for the insightful observation19:19
*** heyongli has quit IRC19:20
*** heyongli has joined #openstack-meeting19:20
*** alexpilo_ has joined #openstack-meeting19:20
clarkbanything else re storyboard?19:20
fungino other storyboard news afaik19:20
clarkbOk and to follow up on config management proposed changes please go review https://review.openstack.org/#/c/565550/ and we'll catch up on that in more detail next week.19:21
clarkb#topic General Topics19:21
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:21
clarkbWhy don't we start with a packethost cloud update. We've set max servers to 0 due to the mtu problem.19:21
clarkbI've got a couple changes up to address that in zuul-jobs and devstack-gate for our network overlay setup19:22
*** shaohe_feng has quit IRC19:22
fungiwhere "the mtu problem" is described simply as the interface mtu on instances there is 145019:22
clarkbright it is smaller than we assume (1500) in a few places19:22
fungi(same as in our linaro arm64 cloud, too)19:22
*** shaohe_feng has joined #openstack-meeting19:23
clarkb#link https://review.openstack.org/578146 handle small mtu in devstack-gate19:23
*** cdub has joined #openstack-meeting19:23
clarkb#link https://review.openstack.org/#/c/578153/ handle small mtu in zuul-jobs19:23
clarkbif we can get those in and osa isn't strongly opposed I'd like to turn packethost back on and see if we get more reliable results19:23
*** slaweq has joined #openstack-meeting19:24
*** alexpilotti has quit IRC19:24
mordred++19:25
clarkbI should also write an email to the dev list explaining we can't make these assumptions anymore19:26
corvusthat looks like a good solution and backwards compat, so merging into zuul-jobs should be fine19:26
corvusat least, backwards compat for working setups :)19:26
clarkbcorvus: yup should be backward compat if you already had to yes that19:26
clarkbkolla and tripleo are the other two I'm semi worried about but I think tripleo has done a decent job reconsuming our overaly tooling19:27
clarkbI expect they will get a working setup if we upate our tools19:27
clarkbThe other packethost issue is a bit more osa specific according to logstash and has to do with tcp checksum errors. Possibly related to mtus but osa also explicitly sets some iptables checksum options19:28
corvusthis might be worth a quick email then, just to make sure kolla/tripleo/other folks see the changes?19:29
*** rbartal has quit IRC19:29
clarkbyup, I'll write one up explaining we have morethan one cloud with smaller MTUs now and we hvae to stop assuming 150019:29
*** rbartal has joined #openstack-meeting19:30
*** heyongli has quit IRC19:30
clarkbhopefully it isn't too controversial as it is a problem we've made for ourselves via neutron overlay networking :)19:30
fungiand we could stand to update the testing environment document accordingly19:30
*** heyongli has joined #openstack-meeting19:30
clarkbfungi: I have a pathc up for that19:30
*** alexpilotti has joined #openstack-meeting19:30
fungiahh, see what i miss when i go to get lunch?19:30
clarkblooks like it must've merged its not in my open queue anymore19:31
fungii'm sure i was +2 on that in spirit19:31
clarkbhttps://review.openstack.org/57815919:31
*** shaohe_feng has quit IRC19:32
clarkbOn winterstack naming I followed up with jbryce last week really quickly and the plan he has proposed is that he wants to double check a couple foundation people have had a chance to look it over then he will request we start +1'ing or similar to whittle our list down. He and a bunch of the foundation staff are travling in APAC right now so that may not happen this week19:33
clarkbfeel free to continue adding suggestions if you have new ideas19:33
*** shaohe_feng has joined #openstack-meeting19:33
*** alexpilo_ has quit IRC19:34
clarkbSSL certs for just about everything were updated late last week. Rooters are not longer getting all those emails every day :)19:35
clarkbOne thing we/I discovered in that process is that we can no longer rely on email based verification for the signing requests19:36
corvusoh?19:36
clarkbyup, GDPR fallout is that our registrar for .orgs is not publishing contact info in whois anymore19:36
corvusis there no forwarder?19:36
clarkband despite asking them fairly directly to publish thatdata they don't seem interested. This wasn't a problem for openstack.org as namecheap lets you use hostmaster@openstack.org but for openstackid.org was more problematic19:37
clarkbcorvus: not that I could determine19:37
fungiit appears that the .org tld registry is special in this case19:37
clarkbThe solution I used was to use DNS record based verification19:37
clarkbI created a random string CNAME to a comodo name and then after 20 minutes they checked and found it there and signed the cert19:37
corvusis one cname sufficient for all certs, or did you have to do that for each?19:38
fungi.com and .net (for example) are still publishing technical, anuse and administrative contact e-mail addresses and mailing addresses in the publiv 43/tcp whois19:38
clarkbcorvus: one for each name in a cert19:38
fungis/anuse/abuse/19:38
corvusblech19:38
corvusinformally, what are our latest thoughts about exploring letsencrypt?19:38
corvusany questions/problems i can help answer/research?19:39
clarkbfungi has apparnetly been poking at it. The process for verifying certs with them is basically the same if using DNS19:39
fungii've gotten over my organizational concerns with the isrg members and am trying it out on some of my personal systems. at this point i'm more concerned about the bootstrapping problem19:39
clarkbyou use a TXT record instead of a cname though iirc19:39
clarkbmost of the tooling that seems to exist today assumes root access which bugs me (but is necessary if listening on a privileged port for verification)19:40
*** heyongli has quit IRC19:40
fungiyeah, the acme rfc draft has details, but basically it's a specially-formatted txt record in your domain, _or_ serving a string from a webserver already present at the dns name in question19:40
corvusi've been using apache19:40
*** alexpilo_ has joined #openstack-meeting19:40
*** heyongli has joined #openstack-meeting19:41
corvusany reason not to use http instead of dns?19:41
clarkbcorvus: mostly my concern with it needing root to perform config management tasks out of band of config management19:41
fungithe dns solution presumes orchestrating dns, and the www mechaism assumes you'll do multipstage deployment initially with no ssl cert19:41
clarkbseparately I also think the cert limit per domain could bite us with letsencrypt19:42
corvusfungi: true, it is difficult to convince apache to start with no cert... but maybe we could have config management handle that?  if no cert, don't add the ssl hosts?19:42
fungithe www solution can also be worked around with a proxy sort of setup, but then boils back down to dns orchestration19:42
clarkba wildcard is an option but that reduces segmentation19:42
*** shaohe_feng has quit IRC19:42
fungithough unrelated, we noticed that *.openstack.org is already in use for a cert with the caching proxy service the foundation is contracting for www.openstack.org19:42
corvuswe could put a handler in all of our vhosts for /.well-known/acme-challenge/19:43
fungisomewhat worrisome, and makes me eager to be on our own domain for important things19:43
corvusfungi: yeah19:43
*** shaohe_feng has joined #openstack-meeting19:43
clarkbcorvus: ya we can engotiate the verification exchange ourselves rather than use the published tools19:43
corvusi mean, we put in a lot of work to avoid doing that and then poof19:43
fungithe ubiquitous acme handler solution is compelling, though still means eventually-consistent sice the first pass of configuration management won't bring up working https19:44
*** alexpilotti has quit IRC19:44
corvusclarkb: oh, i just meant that all we need from apache is a non-ssl vhost which support servince static files in /.well-known/acme-challenge/.  then we can use certbot19:44
clarkboh will certbot operate without opening the port itself?19:44
clarkblast I looked that was a requirement iirc and jamielennox had written a tool to avoid that19:45
fungiit operates over https (80/tcp)19:45
corvusfungi: right, so it's mostly config-management complication.  i haven't thought about what that would mean in the new ansible-container space19:45
fungier, over http19:45
fungii suppose we could temporarily copy snakeoil into the certbot cert/key path until it updates those files19:45
corvusfungi: that's an option too19:45
fungiso allowing apache to listen on 443/tcp from the start19:45
*** lpetrut has joined #openstack-meeting19:46
corvusclarkb: aiui, it will write files to, eg, /var/www/www.example.com/.well-known/acme-challenge/ poke its server, get a cert, then write out the key19:46
fungiand just make sure any generic 80->443 redirect we configure has a carve-out for .well-known/acme-challenge19:46
corvuser, write out the signed cert19:46
corvusfungi: ya19:46
corvusclarkb: "certbot-auto certonly --webroot -w /var/www/www.example.com/ -d www.example.com -d example.com"   makes it do that19:47
clarkbcorvus: cool, good to know19:47
fungithough on my newer systems the updated apache certbot module is working fine and basically provides the same19:47
corvus(this is how i use it without giving it any special access, other than write to that directory and /etc/letsencrypt)19:47
corvusfungi: oh neat, didn't know about that19:48
clarkbother than that my only other real concern is rate limits and quota limits19:48
clarkbI know when they first started it was quite limited but I think they are a lot less limited now19:48
clarkblooks like limit is 20 certs per week now and each cert can have up to 100 names19:49
corvus20 certs/week now according to https://letsencrypt.org/docs/rate-limits/19:49
fungiit's more making sure that since we're in the 20 certs range, we don't try to renew them all in the same week19:50
corvusapparently there's an exemption for renewing :)19:50
clarkbthose limits should be fine given what I just refreshed19:50
*** lpetrut has quit IRC19:50
*** alexpilotti has joined #openstack-meeting19:50
*** heyongli has quit IRC19:50
clarkbI did 17 certs with one having 8 names and the rest had 1 name19:50
clarkband we have a handful of certs that weren't expiring soon enough to worry about19:51
*** heyongli has joined #openstack-meeting19:51
fungile didn't allow san at one time, not sure if they've started to do so19:51
clarkbfungi: the link above implies they do now up to 100 per cert19:51
fungibut could imply we need 24 at this point if all distinct certs?19:51
fungiahh, nice19:51
fungii missed that19:52
*** priteau has quit IRC19:52
*** slaweq has quit IRC19:52
*** e0ne has joined #openstack-meeting19:52
corvusif renewing doesn't count (which is how i read it), we're effectively unlimited.  it's a growth limit, not a size limit.19:52
*** shaohe_feng has quit IRC19:53
corvuser, growth rate19:53
clarkbBefore we run out of time I wanted to give a quick kata + zuul update. We've got a fedora 28 job running now that sometimes crashes (presumably due to nested virt problems). I'm also sorting through what they feel are requirements from their end after a bug update last night19:53
fungioh, neat19:53
*** shaohe_feng has joined #openstack-meeting19:53
clarkbfrom my perspective we haven't run into any deal breakers, things that are broken were either already broken or never running in jenkins in the first place19:53
corvusclarkb: where's the f28 job running?19:54
*** gouthamr has joined #openstack-meeting19:54
clarkbcorvus: against kata-containers/proxy on vexxhost VM19:54
*** alexpilo_ has quit IRC19:54
corvusso it still sometimes crashes even in vexxhost?19:54
corvus(i thought vexxhost had the right settings or something)19:54
clarkbcorvus: yes, apparently they have issues with their Jenkins running ubuntu 17.10 as well19:54
clarkbmnaser thinks it is related to kernel versions in the guest19:54
clarkb(newer kernels less stable)19:55
mnaserwe're running the newest centos kernel on the hypervisor19:55
clarkbhe is working with them to debug that since it affects jenkins just as much as zuul19:55
mnaserand noticed 16.04 never crashed, but 17.10+ was crashing19:55
mnasersalvador from kata came up with a reproducer too19:55
*** jamesmcarthur has joined #openstack-meeting19:56
mnaseri gave all the tracebacks and the info necessary but this isn't as much of a zuul/infra issue as it is a kernel, but i'm open to providing whatever they need to solve things :)19:56
*** hemna_ has quit IRC19:56
corvusdowngrading kernel versions is an option too19:57
corvus(i mean, in the job as a pre-playbook)19:57
*** pchavva has quit IRC19:57
corvussince zuul *can* handle in-job test-node reboots19:57
clarkbreally quickly before we run out of time, is there anything else anyone had to bring up?19:57
clarkb#topic Open Discussion19:57
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:57
clarkb(happy to follow up on kata in -infra)19:57
clarkbor in kata-dev19:57
corvusi gave a talk on zuul at openinfradays china.  it was well attended.19:58
fungithanks for doing that!19:59
corvusmy pleasure :)19:59
clarkbWe are out of time. Thank you everyone. I amgoing to grab lunch then will follow up to the mailing lists with the mtu topic and infra ptg item20:00
clarkb#endmeeting20:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"20:00
openstackMeeting ended Tue Jun 26 20:00:26 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-06-26-19.01.html20:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-06-26-19.01.txt20:00
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-06-26-19.01.log.html20:00
*** alexpilo_ has joined #openstack-meeting20:00
clarkbfind us in #openstack-infra or openstack-infra@lists.openstack.org if you want to discuss things further20:00
*** heyongli has quit IRC20:01
*** Shrews has left #openstack-meeting20:01
*** heyongli has joined #openstack-meeting20:01
*** shaohe_feng has quit IRC20:03
*** shaohe_feng has joined #openstack-meeting20:04
*** alexpilotti has quit IRC20:04
*** dustins has quit IRC20:04
*** rossella_s has quit IRC20:04
*** rossella_s has joined #openstack-meeting20:05
*** trandles has joined #openstack-meeting20:06
*** dustins has joined #openstack-meeting20:07
*** alexpilotti has joined #openstack-meeting20:10
*** heyongli has quit IRC20:11
*** heyongli has joined #openstack-meeting20:11
*** awaugama has quit IRC20:12
*** m_ebert has joined #openstack-meeting20:12
*** shaohe_feng has quit IRC20:13
*** alexpilo_ has quit IRC20:14
*** shaohe_feng has joined #openstack-meeting20:14
*** e0ne has quit IRC20:15
*** m_ebert has left #openstack-meeting20:15
*** jamesmcarthur has quit IRC20:17
*** gouthamr has quit IRC20:19
*** raildo has quit IRC20:20
*** alexpilo_ has joined #openstack-meeting20:21
*** heyongli has quit IRC20:21
*** heyongli has joined #openstack-meeting20:22
*** gouthamr has joined #openstack-meeting20:22
*** shaohe_feng has quit IRC20:23
*** alexpilotti has quit IRC20:23
*** shaohe_feng has joined #openstack-meeting20:25
*** gouthamr_ has joined #openstack-meeting20:29
*** gouthamr has quit IRC20:29
*** gouthamr_ is now known as gouthamr20:29
*** alexpilotti has joined #openstack-meeting20:30
*** erming has joined #openstack-meeting20:31
*** heyongli has quit IRC20:31
*** heyongli has joined #openstack-meeting20:32
*** cdub has quit IRC20:33
*** shaohe_feng has quit IRC20:34
*** jamesmcarthur has joined #openstack-meeting20:34
*** shaohe_feng has joined #openstack-meeting20:34
*** alexpilo_ has quit IRC20:34
*** cdub has joined #openstack-meeting20:38
*** alexpilo_ has joined #openstack-meeting20:40
*** heyongli has quit IRC20:42
*** heyongli has joined #openstack-meeting20:42
*** alexpilotti has quit IRC20:44
*** shaohe_feng has quit IRC20:44
*** munimeha2 has quit IRC20:46
*** shaohe_feng has joined #openstack-meeting20:46
*** martial has joined #openstack-meeting20:46
*** alexpilotti has joined #openstack-meeting20:50
*** stevebaker has joined #openstack-meeting20:52
*** heyongli has quit IRC20:52
*** heyongli has joined #openstack-meeting20:52
*** cdub has quit IRC20:53
*** alexpilo_ has quit IRC20:54
*** shaohe_feng has quit IRC20:54
*** shaohe_feng has joined #openstack-meeting20:55
*** m_ebert has joined #openstack-meeting20:56
*** julim_ has quit IRC20:57
martialWelcome Marcus, we will be starting shortly20:58
*** janders has joined #openstack-meeting20:59
martial#start_meeting Scientific-sig21:00
martial#startmeeting Scientific-sig21:00
openstackMeeting started Tue Jun 26 21:00:39 2018 UTC and is due to finish in 60 minutes.  The chair is martial. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: Scientific-sig)"21:00
openstackThe meeting name has been set to 'scientific_sig'21:00
martialGood day everybody21:00
*** alexpilo_ has joined #openstack-meeting21:01
trandleshi martial21:01
m_ebertHello Martial21:01
jandersgood evening, good morning :)21:01
martialToday we are joined by Marcus Ebert who will share with us "Utilizing Distributed Clouds for Compute and Storage"21:02
jandersexcellent, hello Marcus21:02
martialThe links to the video and slide deck are as follow21:02
*** heyongli has quit IRC21:02
m_ebertHello everyone!21:02
martial#link Video https://youtu.be/QAda-ee-9Ko21:02
*** heyongli has joined #openstack-meeting21:03
martial#link Slides https://goo.gl/g9EmWg21:03
*** rbudden has joined #openstack-meeting21:03
*** b1airo has joined #openstack-meeting21:03
martialnow we understand that not everybody has had a chance to review the video21:03
martialas such we want to give Mr Ebert a chance to discuss his slide deck and answer questions21:03
b1airoo/21:03
martial#chair b1airo21:03
openstackCurrent chairs: b1airo martial21:03
martialwelcome Blair21:04
b1airoThanks martial21:04
*** alexpilotti has quit IRC21:04
martialm_ebert: would you like to give us an introduction to the work please?21:04
*** shaohe_feng has quit IRC21:04
m_ebertSure21:05
b1airoI'm feeding the animals/children breakfast, so sorry if I'm a little slow...21:05
b1airoHi m_ebert21:05
m_ebertHi blairo21:05
*** shaohe_feng has joined #openstack-meeting21:06
*** rossella_s has quit IRC21:06
*** verdurin has joined #openstack-meeting21:06
m_ebertWhat I wanted to show in the slide is the system we developed to utilize different clouds, which can be anywhere in the world, by building system that unifies different clouds and types into a single infrastructure21:07
m_ebertIt hides the cloud structure from the users, which only see a single "local" batch system to which they submit their jobs21:07
m_ebertIn addition, since the jobs can now run anywhere and the user has no idea where, we are also working on an data infrastructure that unifies storage space on different endpoints  into a single files system like structure21:08
*** rossella_s has joined #openstack-meeting21:08
m_ebertan overview of how the compute part work is shown on slide 921:09
m_ebertand slide 19/20 how we would like to have the storage part that the compute is using in the future21:10
b1airom_ebert: presumably this works well for high-throughput workloads. What does the job definition contain, and can the system accommodate jobs added dynamically via API?21:10
*** felipemonteiro has quit IRC21:10
*** alexpilotti has joined #openstack-meeting21:11
m_ebertblairo: do you mean with API submitting directly to condor, or which API do you mean?21:11
*** mjturek has quit IRC21:11
*** heyongli has quit IRC21:12
jandersm_ebert: this is excellent work. Regarding storage, is the workflow 1) download 2) process 3) upload results, or is in-place access to data also possible?21:12
*** heyongli has joined #openstack-meeting21:13
b1airoEither I suppose. (I haven't seen the slides yet sorry - just woke up)21:13
*** alexpilo_ has quit IRC21:14
m_ebertjanders: the workflow of the 2 experiments is right now download-process-upload, but in-place access is possible. On slide 22 for example there is an example how the root analysis framework could open files directly over the network, but any other tool that can stream files should work to when it uses http/webdav21:14
*** shaohe_feng has quit IRC21:15
m_ebertblairo: Yes, all jobs that can be submitted to condor will go to VMs. In the job definition, requirements for RAM,CPUs, disk and so on can be defined, and if not then defaults will be used21:16
*** shaohe_feng has joined #openstack-meeting21:16
jandersm_ebert: great! Slide 22 says mounting the whole data federation is "reasonably fast". Do you know the approximate throughput you're getting? (just an order of magnitude will suffice)21:17
*** dustins has quit IRC21:17
m_ebertand if no VMs available that satisfy these requirements, then cloudscheduler will start a new VM21:17
martialfrom your slide deck, this uses HTCondor as the batch server, can communicate to can use Openstack, OpenNebula, Amazon, Microsoft Azure, and Google Cloud, and I would second the question on storage as I see "DynaFed"?21:17
*** trandles has quit IRC21:17
*** zaneb has quit IRC21:17
m_ebertjanders: getting data from a minio instance running on a VM on the same cloud, we see up to 3Gbps (but we use only files that are some GBs large)21:18
jandersm_ebert: Thank you. What do you think is the limiting factor in terms of achieving more throughput? storage bandwidth? WAN bandwidth? Applications (eg TCP window tuning)?21:20
*** alexpilo_ has joined #openstack-meeting21:21
m_ebertmartial: yes, HTCondor is the batch system, and cloudscheduler can use these types of clouds right now and can communicate with it. Then it uses cloud-init to send the condor config to a VM, which then can communicate with the HTcondor server.21:21
*** cmurphy_vacation is now known as cmurphy21:22
m_ebertThe limiting factor I see is the storage system itself in case of minio (data is on a volume mounted on a VM, where minio is just a layer), the cloud network interface to the outside, and all other ongoing activity on the cloud/hypervisor21:23
*** heyongli has quit IRC21:23
*** heyongli has joined #openstack-meeting21:23
b1airoAny issues with condor here? I have not used it for a long time but remember it was sometimes seeming a but slow to get started21:24
m_ebertusing gfalFS on a baremetal system, we get nearly line speed (tested up to 10Gbps interfaces) - but depends largely on where the endpoints are21:24
*** alexpilotti has quit IRC21:24
*** shaohe_feng has quit IRC21:25
m_ebertNo issues in daily production so far. Only when we have a very large number of cores available temporary (>>10,000) then it made problems in the past21:25
*** gouthamr has quit IRC21:25
*** gouthamr has joined #openstack-meeting21:26
m_ebertWell, not cores really, but job slots  (we run mostly single core jobs, so job slots==cores)21:26
b1airoOk, yes that sounds more familiar21:27
*** shaohe_feng has joined #openstack-meeting21:28
*** erming has quit IRC21:29
*** zaneb has joined #openstack-meeting21:29
b1airoWhat sort of length jobs are you running?21:30
*** alexpilotti has joined #openstack-meeting21:31
m_ebertwe run mostly so called pilot jobs, which are just a wrapper to setup the environment for real jobs and then pull in from the experiments server the real payload. pilots can run up to 2days (defined in condor), payload jobs usually between some minutes and up to about 15h)21:32
martialhow much time is spend copying data then in those jobs?21:32
*** heyongli has quit IRC21:33
*** heyongli has joined #openstack-meeting21:33
*** numans_ has joined #openstack-meeting21:34
*** alexpilo_ has quit IRC21:34
b1airoYeah i was meaning the actual payload jobs. Do you know if condor handles short jobs (10s to 1s of seconds) ok?21:34
m_ebertWell, with dynafed it comes mostly from close by storage so it's just some seconds at the beginning of the job. We run mostly 8core VMs, so 8 such jobs will run in parallel21:34
*** shaohe_feng has quit IRC21:35
m_ebertwell, in our case, the condor job is only the pilot since that is all it knows about. But we had misconfigured pilots before which terminated within seconds and then run through hundreds before we stopped it - condor was fine with that21:36
martialdo you have some local vs remote benchmark to share?21:36
m_ebertmartial: when we had to go back to the site SE, the pulling in of data can take up to half hour or timed out, since it allows only a specific number of parallel transfers and send all others in a waiting queue21:36
martial(on similar enough systems) to see how bad the overhead is?21:37
*** numans has quit IRC21:37
m_ebertmartial: not right now, but working on a benchmarking sytem. There will be the CHEP conference in 2 weeks, I should have it by then to present. I can send a link around once I have it ready21:37
*** shaohe_feng has joined #openstack-meeting21:37
martialthanks we can do a follow up then21:38
jandersthat would be great21:38
*** lifeless has joined #openstack-meeting21:38
m_ebertsure, I'll do that21:38
*** alexpilo_ has joined #openstack-meeting21:41
martialany other questions for Mr Ebert?21:41
martialseems to me we have reached a natural stopping point in this conversation for the time being21:41
martialwe invite people to check the video for additional details21:41
martialallow me to thank you again for coming to explain this very interesting solution to us21:42
martialplease follow with us in a few weeks if you have more to add21:42
*** antosh has quit IRC21:43
martialfor the time, please allow me to thank you for accepting to talk to us m_ebert21:43
m_ebertThank you! Also everyone please feel free to send me questions/comments by email later if some come up by reviewing the slides, also if you have free resources somewhere ;-)21:43
*** heyongli has quit IRC21:43
b1airoThanks m_ebert , we'll also share this in next week's meeting which is friendlier to different timezones21:43
m_ebertI'll try to join more often the meetings and let you know when the benchmarks are ready21:43
martialyour email is listed in the slides21:43
*** heyongli has joined #openstack-meeting21:43
m_ebertthanks blairo21:44
martialthank you21:44
*** fnaval has quit IRC21:44
martialand with this, our other topic for today was21:44
martial#topic AOB21:44
*** openstack changes topic to "AOB (Meeting topic: Scientific-sig)"21:44
*** alexpilotti has quit IRC21:44
martialfor once I do not have content for AOB :)21:44
martialb1airo?21:44
b1airoHa!21:44
b1airoHmmmm, no not off the top of my head. But I am a little slow off the mark this morning21:45
jandersthe submission deadline for Berlin is approaching.. I wonder what talks would you guys like to hear?21:45
*** shaohe_feng has quit IRC21:45
verdurinjanders: the talk where someone solves all my controlled-data problems?21:47
*** rcernin has joined #openstack-meeting21:47
jandersverdurin: :)21:47
b1airoYeah that would be pretty wonderful21:47
b1airoEven just a talk that tells me what they all are21:48
*** fnaval has joined #openstack-meeting21:49
b1airoI'd be interested in hearing about how people do general purpose "managed" cloud, i.e., long-lived instances with patching etc21:49
martialthat is a tough one indeed21:50
jandersb1airo: on your RoCE-enabled system, do you install MOFED in the images or are you running upstream mlnx stack?21:50
*** ian_ott has quit IRC21:51
*** alexpilotti has joined #openstack-meeting21:51
*** shaohe_feng has joined #openstack-meeting21:51
b1airoYeah we use MOFED21:51
jandersdo you embed custom repo config in images and manage kernel-dependent packages?21:51
jandersor do you leave that to the users?21:51
b1airoThough it gets installed post launch via Ansible21:51
jandersdo you keep your own MOFED repos?21:52
janders(or use mlnx iso/tgz ?)21:52
*** mriedem has quit IRC21:52
b1airoThe HPC crew who are the primary users of the RoCE enabled stuff do have their own repo for managing updates consistently21:52
b1airoThey use the ISO I think21:53
jandersok!21:53
*** heyongli has quit IRC21:53
jandersI maintain MOFED repos and put excludes in the yum config - that's mostly for the infra, but I'm considering doing something similar for the instances21:54
*** heyongli has joined #openstack-meeting21:54
*** jamesmcarthur has quit IRC21:54
b1airoI think it's mostly ok now. Though I sometimes hear them grumbling about needing to rebuild modules21:54
janderssounds like mlnx_add_kernel_support.sh :)21:54
*** alexpilo_ has quit IRC21:54
jandersmostly works, occasionally causes frustration..21:54
b1airoYeah. Dkms on Ubuntu seems to be fine21:55
jandersit's an interesting lifecycle-related challenge, quite a boutique one though21:55
janderswhat's your motivation for using MOFED as opposed to upstream? better perforamnce? supportability? good practice? all of the above?21:55
b1airoIt's a bugbear across the research cloud21:56
*** shaohe_feng has quit IRC21:56
martialwe have only a short few minutes at this point21:56
*** mriedem has joined #openstack-meeting21:56
*** afazekas has quit IRC21:56
b1airoLots of fairly green users with root access of public machines. Would be nice to give them guardrails and help protect their data21:57
*** shaohe_feng has joined #openstack-meeting21:57
jandersb1airo: +1! :)21:57
*** afazekas has joined #openstack-meeting21:59
martialand on those words, I am about to end our meeting21:59
martialthanks everybody for spending this time with us21:59
b1airoI see scope for a community project in this, but not something I can start at the moment!21:59
b1airoCheers all21:59
martialand to Mr Ebert for presenting21:59
martialbye all21:59
martial#endmeeting21:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"21:59
openstackMeeting ended Tue Jun 26 21:59:53 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-06-26-21.00.html21:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-06-26-21.00.txt21:59
m_ebertThanks everyone for the discussion!  cheers21:59
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-06-26-21.00.log.html22:00
*** m_ebert has quit IRC22:00
*** alexpilo_ has joined #openstack-meeting22:01
*** janders has quit IRC22:01
*** heyongli has quit IRC22:04
*** edmondsw has quit IRC22:04
*** heyongli has joined #openstack-meeting22:04
*** alexpilotti has quit IRC22:04
*** shaohe_feng has quit IRC22:06
*** edmondsw has joined #openstack-meeting22:07
*** bobh has quit IRC22:07
*** shaohe_feng has joined #openstack-meeting22:08
*** felipemonteiro has joined #openstack-meeting22:10
*** alexpilotti has joined #openstack-meeting22:11
*** edmondsw has quit IRC22:11
*** antosh has joined #openstack-meeting22:12
*** felipemonteiro_ has joined #openstack-meeting22:12
*** felipemonteiro has quit IRC22:12
*** felipemonteiro__ has joined #openstack-meeting22:13
*** alexpilo_ has quit IRC22:14
*** heyongli has quit IRC22:14
*** heyongli has joined #openstack-meeting22:14
*** shaohe_feng has quit IRC22:16
*** shaohe_feng has joined #openstack-meeting22:16
*** felipemonteiro_ has quit IRC22:17
*** alexpilo_ has joined #openstack-meeting22:21
*** mriedem has quit IRC22:21
*** alexpilotti has quit IRC22:24
*** heyongli has quit IRC22:24
*** heyongli has joined #openstack-meeting22:24
*** shaohe_feng has quit IRC22:26
*** shaohe_feng has joined #openstack-meeting22:27
*** bnemec has quit IRC22:29
*** alexpilotti has joined #openstack-meeting22:31
*** rbudden has quit IRC22:34
*** alexpilo_ has quit IRC22:34
*** heyongli has quit IRC22:34
*** bnemec has joined #openstack-meeting22:35
*** heyongli has joined #openstack-meeting22:35
*** hongbin has quit IRC22:36
*** shaohe_feng has quit IRC22:37
*** shaohe_feng has joined #openstack-meeting22:37
*** alexpilo_ has joined #openstack-meeting22:41
*** Swami_ has quit IRC22:43
*** Swami has quit IRC22:43
*** alexpilotti has quit IRC22:44
*** heyongli has quit IRC22:45
*** heyongli has joined #openstack-meeting22:45
*** shaohe_feng has quit IRC22:47
*** shaohe_feng has joined #openstack-meeting22:47
*** rbudden has joined #openstack-meeting22:47
*** alexpilotti has joined #openstack-meeting22:51
*** alexpilo_ has quit IRC22:54
*** tpsilva has quit IRC22:55
*** heyongli has quit IRC22:55
*** heyongli has joined #openstack-meeting22:55
*** rossella_s has quit IRC22:56
*** shaohe_feng has quit IRC22:57
*** rossella_s has joined #openstack-meeting22:59
*** shaohe_feng has joined #openstack-meeting22:59
*** rbudden has quit IRC23:00
*** alexpilo_ has joined #openstack-meeting23:01
*** felipemonteiro__ has quit IRC23:01
*** alexpilotti has quit IRC23:04
*** Kevin_Zheng has joined #openstack-meeting23:05
*** heyongli has quit IRC23:05
*** heyongli has joined #openstack-meeting23:05
*** shaohe_feng has quit IRC23:07
*** shaohe_feng has joined #openstack-meeting23:08
*** bobh has joined #openstack-meeting23:08
*** alexpilotti has joined #openstack-meeting23:11
*** alexpilo_ has quit IRC23:15
*** heyongli has quit IRC23:15
*** heyongli has joined #openstack-meeting23:16
*** shaohe_feng has quit IRC23:18
*** shaohe_feng has joined #openstack-meeting23:21
*** alexpilo_ has joined #openstack-meeting23:21
*** alexpilotti has quit IRC23:24
*** heyongli has quit IRC23:26
*** heyongli has joined #openstack-meeting23:26
*** shaohe_feng has quit IRC23:28
*** shaohe_feng has joined #openstack-meeting23:29
*** alexpilotti has joined #openstack-meeting23:31
*** alexpilo_ has quit IRC23:34
*** mjturek has joined #openstack-meeting23:35
*** heyongli has quit IRC23:36
*** mlavalle has quit IRC23:36
*** chyka has quit IRC23:36
*** heyongli has joined #openstack-meeting23:36
*** rpioso is now known as rpioso|afk23:36
*** shaohe_feng has quit IRC23:38
*** shaohe_feng has joined #openstack-meeting23:39
*** antosh has quit IRC23:39
*** alexpilo_ has joined #openstack-meeting23:41
*** alexpilotti has quit IRC23:44
*** tetsuro has joined #openstack-meeting23:45
*** heyongli has quit IRC23:46
*** heyongli has joined #openstack-meeting23:47
*** shaohe_feng has quit IRC23:48
*** shaohe_feng has joined #openstack-meeting23:49
*** alexpilotti has joined #openstack-meeting23:51
*** mjturek has quit IRC23:54
*** alexpilo_ has quit IRC23:54
*** heyongli has quit IRC23:56
*** mjturek has joined #openstack-meeting23:57
*** heyongli has joined #openstack-meeting23:57
*** shaohe_feng has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!