Wednesday, 2017-06-21

*** amotoki_away is now known as amotoki00:23
*** cuongnv has joined #openstack-fwaas00:59
*** SridarK has joined #openstack-fwaas02:06
SridarKreedip: hi02:06
SridarKsorry just got home02:07
reedipHi02:07
reedipI just JUST reached office02:07
SridarKdid u guys start the discussion02:07
SridarKah ok02:07
reedipno , but I think yushiro was also going to join02:07
xgerman_hi02:08
reedipwe are also missing yamamoto02:08
SridarKso can u summarize the issue for me02:08
SridarKxgerman_: hi02:08
reediphi xgerman_02:08
reedipso this is the issue02:08
SridarKso basically once we moved the fwaas api into neutron-lib02:08
SridarKalthough we seem to be fine02:08
SridarKmidonet seems to have some failures02:09
reedipWhen we moved to neutron-lib's definition to https://review.openstack.org/#/c/456511/02:09
SridarKis that an accurate understanding ?02:09
reedipmidonet started failing02:09
reediphttps://review.openstack.org/#/c/471699/02:09
reedipAs per yamamoto san's comments, The change seems to have a confusion between02:09
reedipservice type constant ("FIREWALL") and the extension. ("fwaas")02:09
reedipso before we moved to neutron-lib, we were using FIREWALL as a constant to define the service02:10
SridarKyes02:10
reedipnow if we see the base file in https://review.openstack.org/#/c/456511/22/neutron_fwaas/extensions/firewall_v2.py , you will find in line#32, we are using FIREWALL02:11
reedipthis is also defined as a constant in https://review.openstack.org/#/c/456511/22/neutron_fwaas/common/fwaas_constants.py02:12
reedipbut during migration to neutron-lib, the FIREWALL value was converted to fwaas02:12
SridarKi think the ext alias02:13
SridarKwill return fwaas02:13
reedipAs per yamamoto, the unit test took care of the current predicament by changing fwaas to FIREWALL https://review.openstack.org/#/c/456511/22/neutron_fwaas/tests/unit/services/firewall/test_fwaas_plugin.py@10202:13
reedipsuch changes probably allowed FWaaS Unit tests to pass, and were in line with the changes made in the extension02:14
SridarKyes this is the service type to load the correct plugin02:15
reedipEven after the service name constant was reverted in https://review.openstack.org/#/c/471720/ , the unit tests didnt fail , although now the unit tests accepted FIREWALL instead of fwaas02:15
SridarKdo u happen to have a pointer to the midonet logs02:16
SridarKwith the failures02:16
reedipSridarK, and infact if we follow https://review.openstack.org/#/c/471720/ , midonet tests right now pass02:16
reedipif we execute the tests serially02:16
SridarKreedip: yes as u mentioned in the mtg02:16
SridarKnow that is puzzling02:16
reedipSorry, wrong link above02:17
SridarKin fact if this is a problem - i am pulling my hair (not that there is much there now)02:17
SridarKhow does serial exec pass02:17
reedipSridarK , xgerman_ : The logs here state that Midonet passes the gate02:18
reediphttp://logs.openstack.org/16/443416/2/check/gate-networking-midonet-python27-ubuntu-xenial/5540ce8/console.html.gz02:18
SridarKdo u happen to have a pointer where it failed02:18
xgerman_so non issue?02:18
reedipSrdiarK , xgerman_ Parallel Execution fails !02:19
reedipserial doesnt02:19
xgerman_mmh, so it’s order dependant?02:20
SridarKthat could be02:21
SridarKwhen it fails, which test fails ?02:21
reedipxgerman_ could be02:21
SridarKreedip: v1 or v2 - any consistent pattern02:21
reedipSridarK : not defined. But you can easily replicate it.02:21
SridarKok02:22
reedipYou can run py27 Midonet tests for fwaas and you will see the errors coming as of now02:22
reedipits mainly in v102:22
SridarKSo i clone midonet02:22
reedipSridarK , xgerman : What me and yushiro noticed was something like this : http://logs.openstack.org/16/443416/2/check/gate-networking-midonet-python27-ubuntu-xenial/5540ce8/console.html.gz#_2017-06-08_06_36_52_19242702:22
SridarKand do run py2702:23
reedipDuplicate extension is already loaded and therefore the extension is not reloaded02:23
reedipSridarK : yes, that would work02:23
reedipSridarK : you can also run fwaas specific tests like02:23
reedip" tox -v -e py27 midonet.neutron.tests.unit.test_extension_fwaas"02:24
reedipThat would reduce the number of test cases and generally fails in my checkout02:24
SridarKyes duplicate ext would be a prob02:24
SridarKreedip: and the repo is ?02:26
SridarKsorry i have never tried anything in midonet02:26
reedipSridarK : IIUC the situation, it states that the extension is already loaded and therefore doenst load it again, but the loaded extension is actually not present. The false positive fails to load the extension and thus fails the extension related test cases02:26
reedipSridark : just clone networking-midonet02:26
SridarKah ok02:26
*** yushiro has joined #openstack-fwaas02:26
SridarKok let me try some tonight02:27
SridarKmy time02:27
yushiromorning.  So sorry for late.02:27
SridarKyushiro: hi02:27
yushiroSridarK, Hi, good day02:27
SridarKi am not opposed to the revert02:27
SridarKbut will be good to have some data02:27
SridarKreedip: do u think another day will cause a lot of pain and issue ?02:28
xgerman_we had it sitting for a while so a day more shouldn’t be an issue02:28
SridarKok02:28
reedipok02:29
reedipxgerman_ SridarK , I agree, we can keep it for one more day02:29
reediphi yushiro , we were just discussing the problem at hand02:29
SridarKreedip: thx for the context02:29
SridarKlet me look some02:30
SridarKand will let u know02:30
reedipSridarK : yeah sure02:30
SridarKi will put a comment on gerrit so yamamoto is aware02:30
reedipok02:30
yushiroreedip, OK.  a cause was loading twice fwaas extension, wasn't it?02:30
reedipyushiro : IIUC the situation, it states that the extension is already loaded and therefore doenst load it again, but the loaded extension is actually not present. The false positive fails to load the extension and thus fails the extension related test cases . The same has been conveyed in the meeting02:30
SridarKwe can discuss tomorrow and then if there is no obvious issue02:30
SridarKsorry no obvious solution - we can revert02:31
reedipyushiro : yes, seems to be that. On serial execution the extension is not loaded twice.02:31
xgerman_fwaas loaded twice is fishy… sometimes those tests’ inheritance is pretty circular02:31
reedipbut seems to be loaded twice on parallel execution02:31
reedipxgerman_  : any tool to find cross inheritance in the code without going too deep ?02:31
xgerman_sorry, I just use pycharms “Find Definition” repeatedly02:32
reedipwhat you say may be possible because of the test cases loading up the same file twice in parallel execution02:32
reediphow do we use that xgerman_ ?02:32
xgerman_I remember when I added something I had to be pretty surgical in overriding02:33
xgerman_well, it just goes to the superclass’s definition and you can trace it by hand02:33
SridarKand also why we dont see it in the fwaas repo is another thing02:34
reedipSridarK : I saw it earlier in the fwaas repo but the Unit test change may have hidden it02:35
SridarKhmm oh u did02:35
reedipI did see it02:35
SridarKwhat u pointed earlier L#102 ?02:35
SridarKreedip: ^^02:35
reedipbut after making the Unit test plugin similar to the firewall extension, the issue didnt occur02:35
reedipSridarK : yes02:35
SridarKok02:36
SridarKthx reedip tht is good info02:36
xgerman_+102:37
reedipI think this issue may b visible in https://review.openstack.org/#/c/421472/29 right now as well, if I am correct02:38
reedipthis was the older patch from Nate02:38
xgerman_k02:39
SridarKok02:39
SridarKlet me do some digging too a bit later tonight02:40
reedipyushiro , if you have time, lets discuss more on this. SridarK, xgerman_ I guess its already late for you so if you would like to continue the discussion today evening ( our side) /tomorrow morning( your side ) , then let us know02:40
reedipxgerman_ : I will try the pycharms02:40
yushiroreedip, sure.02:40
SridarKreedip: yes that is good idea02:40
SridarKi will step away for sometime but will look at the logs02:41
yushiroNow I'm digging into midonet/neutron/tests/unit/test_extension_fwaas.py02:41
SridarKand also shoot me an email if u find something02:41
reedipok ...02:41
SridarKthx02:41
xgerman_same —  I will check back tomorrow morning…02:41
reediplets put our points in the email  and on this channel02:41
yushirothanks reedip02:43
*** yushiro is now known as yushiro_lunch03:02
*** SarathMekala has joined #openstack-fwaas03:05
*** SridarK has quit IRC03:44
*** yushiro_lunch is now known as yushiro03:48
*** yamamoto has joined #openstack-fwaas03:57
reedipyushiro , hi. It seems midonet is not able to get the proper plugin for Firewall in midonet/neutron/services/logging_resource/plugin.py04:32
reedipLine#17304:32
yushirofw_plugin = directory.get_plugin(const.FIREWALL) f_log_info['firewall'] = fw_plugin.get_firewall(context, f_log_info['firewall_id'], fields=['id', 'tenant_id'])04:35
yushirodirectory.get_plugin(const.FIREWALL)04:35
yushirois fishy04:35
yushiro+104:36
reedipyamamoto is online, maybe he can also help ?04:36
*** SarathMekala has quit IRC04:37
yushiroyamamoto, ping04:37
reedipyushiro : can we check if extension is loaded , before the plugin loads the extension ?05:04
amotokiyushiro: do you see +2 right in neutron-fwaas-dashbaord like https://review.openstack.org/#/c/475965/ ?05:05
yushiroreedip, I'll try it.  Just a moment.05:05
amotokiyushiro: I expect I see +2 button as horizon-core but I don't see +2 button though.... so i would like some others to check it.05:05
yushiroamotoki, just a moment...05:06
yushiroamotoki, Yes, I have +2 right.05:06
amotokiyushiro: hmm...  what is happening to me.... self-patch???05:06
yushiroamotoki, If this patch is self-patch, you can put +2...  I could it in networking-fujitsu repos.05:07
yushirothat's strange situation..05:07
amotokiyushiro: I think so too. we can usually put +2 to self-patch05:08
amotokiyushiro: could you approve that one? it is simple enough and part of initial setup works. there is no problem.05:08
amotokiyushiro: i will ask it in the infra channel.05:08
yushiroamotoki, OK.05:09
yushiroamotoki, +A is OK?05:09
amotokiyushiro: yes, please05:09
yamamotoyushiro: pong05:09
yushiroamotoki, done05:10
amotokiyushiro: thanks.05:11
amotokiyushiro: in addition, I suddenly see +2 button now. somehow strange, but it is okay now.05:11
yushiroamotoki, that's fine :)05:11
yamamotoyushiro: reedip: i read the back log but i'm not sure what i can help. can you explain?05:13
*** cuongnv has quit IRC05:14
yamamotoit's more than two weeks.  can you consider to revert?  https://review.openstack.org/#/c/471699/05:16
yushiroyamamoto, still discussing.  Currently, I'm reading networking-midonet and test result.  Maybe it loads twice for fwaas extension.05:18
yamamotobtw, we're going to disable tests to unblock the gate.  https://review.openstack.org/#/c/475245/05:18
yamamotodiscussing what?05:18
*** cuongnv has joined #openstack-fwaas05:19
yamamotoi don't see any point to keep it broken.05:20
yushiroah, reverting patch will approve it but need to know why fails networking-midonet.05:23
reedipyamamoto : we agree that keeping it broken doenst make sense but we are trying to find the cause of the issue. We would likely revert the whoile change but we also need to push for migration to neutron-lib.05:23
reedipIn order to avoid the same issue to occur again, we would like to knoe why networking-midonet fails05:24
yushirowe're discussing what is the next approach after merging reverting patch.05:24
yushiroyamamoto, yes.  reedip says what I'd like to say.05:24
yamamotowhat's wrong with reverting it, and then investigate?05:24
reedipyamamoto : nothing wrong. The revert will probably get the votes today evening. we have discussed the same in yesterday's meeting05:25
reedipwe are inclined to fix the midonet issue soon, do not get us wrong :)05:26
yamamotook05:26
yamamotowrt why it fails,05:26
yamamoto1. service type mismatch05:27
yamamoto2. race in midonet reedip mentioned (i'm not sure what it is)05:27
yamamotoanything else known?05:27
reedipyamamoto : the directory.get_plugin( 'FIREWALL') returns None in  midonet/neutron/services/logging_resource/plugin.py05:30
reedipWe are looking at that as well05:30
yushirohttps://github.com/openstack/networking-midonet/blob/master/midonet/neutron/services/logging_resource/plugin.py#L17205:30
yamamotoit should be a consequence of the service type mismatch05:31
yushiroyamamoto, https://github.com/openstack/neutron-lib/commit/1331fb208330d1ca385996a716f0bc0c7ad1a07f05:35
yushiroboden has committed into neutron-lib as plugin common constants05:36
yushirothis is 'FIREWALL'.  Previous name was 'fwaas'.  That's a service type mismatch I think.  So, in order to recover networking-midonet, should we modify service type from 'FIREWALL' to 'fwaas' ?05:37
yushiroI'd like to know a next approach for case "1"05:39
yamamotoyushiro: no. FIREWALL is correct.  service type != extension alias05:48
amotoki'FIREWALL' is used to look up service plugin or something. extension alias is used in the API.05:49
yushiroyamamoto, amotoki OK, thank you.  I confused.06:04
*** trungnv has quit IRC06:18
*** trungnv has joined #openstack-fwaas06:21
reedipamotoki, yamamoto : do you have any idea how the Service type matches the extension ? I mean how are they related ?07:09
yamamotoreedip: see EXT_TO_SERVICE_MAPPING07:11
reedipyamamoto : this is https://github.com/openstack/neutron/blob/b40ce0759f2c2e8d957813b54146f56fb1a19cbe/neutron/plugins/common/constants.py#L3107:12
reedipit maps the extension fwaas to the service FIREWALL07:12
yamamotoyes07:13
reedipyamamoto : but this doesnt map fwaas_v2 and firewallrouterinsertion07:16
yamamotoit isn't available for every extensions. right.07:17
reedipyeah, because its valid only for core extension07:17
reedipyamamoto : the plugin is loaded by https://github.com/openstack/neutron/blob/aeee7a5c908122074dc9c5117d5a5747fc4784d2/neutron/manager.py#L17307:18
yamamotothe term "core extension" is usualy used for something different.07:18
reedipyamamoto : can you explain what is meant by core extension ?07:18
reedipit would help me understand better07:19
yamamotoextension for core resources07:19
yamamotocore resources are network, subnet, port07:19
reedipso network, router, ports etc07:19
yamamotoiirc router is not07:19
amotokigenerally speaking, there are neutron plugins (one core plugin + one+ service plugins) and each plugin needs to declare which extension it implements.07:23
amotokione extension must be implemented by only one plugin.07:23
amotokiMAP_TO_SERVICE_MAPPING is originaly prepared to check it.07:24
amotokiI think this logic can be improved to more simple one, but it is what we have now.07:25
reedipok amotoki, yamamoto, thanks07:31
reediplet me look at the issue further.07:31
reedipyushiro  : as per https://github.com/openstack/neutron-lib/blob/fc38a3fe8e6fa7c75572180554a02b43a72f9af9/neutron_lib/plugins/directory.py#L40 , it seems that midonet doesnt know if the Alias "FIREWALL" exists08:08
reedipthats why its returning NONE08:08
yushiroreedip, aha. So, it's better to refer ALIES in neutron-lib directly, isn't it?08:09
yushiroa,08:09
reedipyushiro : the thing is, midonet is referring the neutron-lib constants for the alias08:10
reedipLet me check one thing , wait08:10
yushiroyes. sorry I confused again ;(08:10
reedipyushiro : not more than me :)08:12
reedipthe extension-service name combination is not easy08:13
yushiroALIAS = 'fwaas'08:16
reedipyushiro : that actually works08:19
reedipyushiro : not sure why08:20
yushiroIn order to refer plugin, it should be executed in advance.  directory.add_plugin(fw_const.FIREWALL, fw_plugin)08:23
yushiroI think.08:23
yushirofor testing...08:24
yushiroreedip, you said 'race' is such as loading plugin ?08:24
yushiros/is/was08:25
reedipyushiro : yes08:25
reedip     WARNING [neutron.api.extensions] Extension file firewall.py wasn't loaded due to Found duplicate extension: fwaas.08:25
reedip     WARNING [neutron.api.extensions] Extension file firewallrouterinsertion.py wasn't loaded due to Found duplicate extension: fwaasrouterinsertion.08:25
*** lnicolas has quit IRC08:25
yushiroAccording to this warning msg, I thought directory.add_plugin() called twice08:26
reedipyushiro : me too08:27
yushiroreedip, ah... just a moment.  https://github.com/openstack/neutron-lib/blob/fc38a3fe8e6fa7c75572180554a02b43a72f9af9/neutron_lib/plugins/directory.py#L3308:28
reedipyushiro  , yamamoto  : let me push a patch. I think the change in the ALIAS of Firewall extension is a reason for the failure08:28
yushiroself._plugins is just a dict.  If same 'key' is registered twice, it result in08:28
reedip??08:29
yushiro1 key.08:29
reedipyushiro : but the thing is we are expecting an ALIAS08:29
yushiroyes... sorry, my brain is now heating ;p08:31
reedipyushiro : relax08:32
reedipyushiro :as you stated earlier, boden changed the ALIAS from fwaas to FIREWALL in neutron-lib08:32
reedipnow midonet refers the neutron-lib constants  to fetch the alias, to map it with a dictionary08:33
reedipto load the plugin08:33
yushiroOK.08:33
reedipfirewall extension still illustrates the ALIAS as fwaas08:33
reedipand not firewall08:33
reedipwhen boden changed fwaas to FIREWALL08:34
reedipthere was no change in midonet08:37
reedipbut when firewall picked up the neutron-lib definition , a lot of changes were incorporated08:37
yushiroOK, I understood 'fwaas' -> 'FIREWALL' has no effect.08:39
yushiroHmm, migrating a lot of definitions has some effect to networking-midonet.08:39
reedipyushiro:  now midonet also uses the fwaas_constants08:39
reedipso changes done there also impacted midonet08:39
reedipthe revert of the service name did not hide the effect of the alias name change as done by boden.08:40
reediphttps://review.openstack.org/#/c/456511/22/neutron_fwaas/extensions/firewall.py@81 loaded firewall as fwaas08:41
reedipif you see https://github.com/openstack/neutron/blob/7e9986411c48cc4302910beaec159ec9bb6aa558/neutron/api/v2/resource_helper.py#L47, it states that the service name should not be fwaas but FIREWALL08:42
reedipsee the third argument to build_resource_info08:42
yushiroreedip, OK, I'll look.08:45
reedipyushiro : I republished the patch https://review.openstack.org/44341608:47
yushiroOK,08:49
yushirothanks08:49
yushiroIf this PS has passed, we should revert a patch and merge your service name change patch.08:50
reedipyushiro : this patch will fail but the failure may not be because of fwaas, but because of the load-synthetic-db issue already existing in midonet.08:50
reedipyushiro : If everything goes fine08:51
reedipa) We need to merge the Service Name revert08:51
reedipb) We need to merge the midonet firewall exception migration08:51
reedipc) We need to merge this patchset ( but I think this fix can be moved to the migrate firewall exception  and this patch can just remain for testing)08:52
yushiroOK.08:54
yushiroSo, after that, we should migrate all of definitions into neutron-lib finally.08:54
reedipYes,08:55
reedipOk, I am now taking a break, and waiting for theresults of this issue08:57
yushiroOK,  I'll go for dinner now.08:59
*** yushiro is now known as yushiro_dinner08:59
reedipok :)09:03
*** yamamoto has quit IRC09:22
*** qiwei has quit IRC09:25
doudeHi all! Just a question on the FWaaS v2 model. A port can be associated to only one fwg at a time?09:35
yushiro_dinnerdoude, yes. Only 1 firewall group can be associated with a port.09:41
*** yushiro_dinner is now known as yushiro09:41
doudewhy this limitation?09:41
yushirodoude, port : firewall group = 1 : 1 ,  firewall_group : port = 1 : n09:41
yushirodoude, If we support multiple firewall_groups for a port, considering an order for firewall group is necessary and it's too complexity.09:44
doudeok09:44
yushirodoude, that's why we should support 1 firewall group per port.09:44
doudethanks yushiro09:44
yushirodoude, np.09:44
*** cuongnv_ has joined #openstack-fwaas09:45
reedipyushiro : what about the firewall-policy to firewall group associattion09:46
*** yamamoto has joined #openstack-fwaas09:46
yushiroreedip, fwg and fwp = 1:2  for ingress/egress.09:47
*** cuongnv__ has joined #openstack-fwaas09:47
*** cuongnv has quit IRC09:48
reedipyushiro : you reported this : https://review.openstack.org/#/c/370731/09:48
reediphttps://launchpad.net/bugs/162309909:48
openstackLaunchpad bug 1623099 in neutron "FWaaSv2 - 'firewall_policy_id' is missing in firewall_rule response body" [Low,New] - Assigned to Reedip (reedip-banerjee)09:48
*** cuongnv__ is now known as cuongnv09:49
*** cuongnv_ has quit IRC09:50
yushiroreedip, In my memory, in v2, firewall_rule can be associated with any firewall_policies.09:52
reedipbut thats missing in the API )09:52
*** yamamoto_ has joined #openstack-fwaas10:09
*** yamamoto has quit IRC10:13
yushiroreedip, In my early thinking, I filed above bug report.  But....10:13
yushiroreedip, I'll talk about next IRC meeting.(not sure in my memory ;))10:14
*** yamamoto_ has quit IRC10:30
*** yamamoto has joined #openstack-fwaas10:32
*** yamamoto has quit IRC10:33
*** cuongnv has quit IRC10:39
*** yamamoto has joined #openstack-fwaas10:43
*** yamamoto has quit IRC10:54
*** yamamoto has joined #openstack-fwaas11:24
*** yamamoto has quit IRC11:28
*** bzhao has quit IRC11:47
*** yushiro has quit IRC11:48
*** bzhao has joined #openstack-fwaas11:48
*** yamamoto has joined #openstack-fwaas11:54
*** yamamoto_ has joined #openstack-fwaas16:10
*** yamamoto has quit IRC16:13
*** bzhao has quit IRC16:19
*** bzhao has joined #openstack-fwaas16:20
*** yamamoto_ has quit IRC16:47
*** yamamoto has joined #openstack-fwaas16:49
*** Tim_Eberhard has joined #openstack-fwaas17:20
*** yamamoto has quit IRC17:42
*** yamamoto has joined #openstack-fwaas17:51
*** yamamoto has quit IRC17:56
*** yamamoto has joined #openstack-fwaas18:15
*** yamamoto has quit IRC18:44
*** yamamoto has joined #openstack-fwaas19:45
*** yamamoto has quit IRC19:52
*** amotoki is now known as amotoki_away20:52
*** Tim_Eberhard has quit IRC22:24
*** Tim_Eberhard has joined #openstack-fwaas22:26
*** lnicolas has joined #openstack-fwaas22:29

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!