MicroWebserver

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

MicroWebserver

Brad Johnson
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Christian Schneider
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
I'll have to look at the Aries RSA as I'm not as aware of it.  It doesn't appear to be similar to DOSGi from my experience with it.  I'll take a look at them again tomorrow.  Like so many things in our OSGi, ServiceMix, Camel world there are about 10 different ways to do everything.  Some are better than others and most are better in certain circumstances.  This design is in reaction to consulting on a job that involves a company which is integrating multiple smaller companies that it owns and they all have a little different security, different ports, different web service requirements but are commonly going to be talking to the same Fuse instance in many (but not all) cases.


 This uses the OSGi mechanics for registration only and it doesn't require that the the interface itself be an OSGi service.  Here is a listener on a blueprint file for the MicroWebserviceManager.  I'm not filtering on any service property but but I certainly could.  The manager has a configuration associated with it for standard interceptors, etc. 


<reference-list id="microserviceListener" interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<reference-listener bind-method="register" unbind-method="unregister">
<ref component-id="microserviceManager" />
</reference-listener>
</reference-list>


The simple Echo bundle I have in there has this in its blueprint.

<service interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<bean class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
<argument value="org.enjekt.osgi.api.echo.EchoService" />
<argument ref="impl" />
<property name="restRelativeURI" value="/resources/echo" />
<property name="soapRelativeURI" value="/services/echo" />
                      
</bean>
</service>

That MicroWebServiceRegistration inherits from the same class that the .MicroWebserviceManager uses for configuration.  When this simple Echo bundle starts it registers as a MicroWebService and the manager picks it up and combines the configuration found in both.  As you can see registration isn't specifying the host/port information so the one used by the MicroWebserviceManager will be used.  However, if I did specify a different host/port in the bundle's properties that would be used instead.  The manager itself never actually creates the server/endpoints.  It instantiates a container that it keeps in a map and that container instantiates the server/endpoints based on the combined information.  

Obviously if I uninstall the Echo bundle this particular MicroWebservice is unregistered, the manager is notified and it tells the container to shut down the endpoint.  Those hard coded properties for restRelativeURI and soapRelativeURI would normally be part of the blueprint properties and so settable at runtime triggering a reload.  That in turn triggers the manager to tear down the old endpoint and start up a new one.  Since the MicroWebserviceRegistration can accept the same sort of interceptors and providers that the manager uses I've used that as a way to register error/exceptionhandlers as well and in one case to override the JSON provider configuration.  One department wanted badgerfish and another wanted the root stripped and unwrapped.  Just specifying those differently in the sending bundle I could override the defaults.  If I wanted to I could also use different registrations to expose the same service on different relativeURIs or ports with different configuration information.  Here's an example of a service adding in an exception handler.

                <property name="addProvider">
                            <bean class="com.foo.internal.exceptionhandlers.MyExceptionHandler" />
                  </property

If that's added to the manager's configuration then all endpoints that register with it get that exception handler.  If it is registered with the bundle's MicroWebserviceRegistration then that only applies to it.  In this case it is in an internal package but gets provided via the web service.  The MicroWebserviceRegistration also provides a UUID so when it the bundle is uninstalled and the manager gets notified of the fact it knows which container to pull from its registry and shut down.

Just looking at that EchoService if I have three different divisions with different security and REST/SOAP requirements I can have three managers running via a simple blueprint set up of their service managers with service property filters for company A, company B, company C and when I register the EchoService I can specify different relative URIs but let each of the managers determine how the interface is actually going to be exposed.

It works very well.  In practice I have most of the properties relevant to the bundle's requirements exposed in the cfg files making changes simple.

I'll take a look at RSA tomorrow and see how it works.

Brad

On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider <[hidden email]> wrote:
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Christian Schneider
Thanks for the explanations. I think I now understand better.

In effect it does a similar thing but with a different focus. Aries RSA and the CXF provider focus on the whiteboard pattern to kind of transparently export/import a service. Both do not have  strong out of the box support for managing and centralizing configurations. It can be done but it less prepared than in your framework.
This is an example of how to export a rest service:
https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

Your framework is more similar to the CXF blueprint namespace as you have to explicitly export a REST or SOAP service using blueprint. It focuses on the configuration aspect by managing a system wide and per service configuration. Something l really like there is the do a lot of the config system wide this is something I would like to add for Aries RSA too.

I really like the whiteboard approach of RSA as it allows to have no or minimal dependencies from the user code to the framework. So maybe you are interested in combining the capabilities of both.
RSA now has a custom service provider interface ExportPolicy. This is a hook where you can do system wide configuration and also things like auto exporting services based on annotations.

I think you can achieve the same level of manageability in RSA as with your framework but it is not there already. If you are interested in helping out there I can help you to get into the code.

Christian

On 17.04.2016 01:08, Brad Johnson wrote:
I'll have to look at the Aries RSA as I'm not as aware of it.  It doesn't appear to be similar to DOSGi from my experience with it.  I'll take a look at them again tomorrow.  Like so many things in our OSGi, ServiceMix, Camel world there are about 10 different ways to do everything.  Some are better than others and most are better in certain circumstances.  This design is in reaction to consulting on a job that involves a company which is integrating multiple smaller companies that it owns and they all have a little different security, different ports, different web service requirements but are commonly going to be talking to the same Fuse instance in many (but not all) cases.


 This uses the OSGi mechanics for registration only and it doesn't require that the the interface itself be an OSGi service.  Here is a listener on a blueprint file for the MicroWebserviceManager.  I'm not filtering on any service property but but I certainly could.  The manager has a configuration associated with it for standard interceptors, etc. 


<reference-list id="microserviceListener" interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<reference-listener bind-method="register" unbind-method="unregister">
<ref component-id="microserviceManager" />
</reference-listener>
</reference-list>


The simple Echo bundle I have in there has this in its blueprint.

<service interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<bean class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
<argument value="org.enjekt.osgi.api.echo.EchoService" />
<argument ref="impl" />
<property name="restRelativeURI" value="/resources/echo" />
<property name="soapRelativeURI" value="/services/echo" />
                      
</bean>
</service>

That MicroWebServiceRegistration inherits from the same class that the .MicroWebserviceManager uses for configuration.  When this simple Echo bundle starts it registers as a MicroWebService and the manager picks it up and combines the configuration found in both.  As you can see registration isn't specifying the host/port information so the one used by the MicroWebserviceManager will be used.  However, if I did specify a different host/port in the bundle's properties that would be used instead.  The manager itself never actually creates the server/endpoints.  It instantiates a container that it keeps in a map and that container instantiates the server/endpoints based on the combined information.  

Obviously if I uninstall the Echo bundle this particular MicroWebservice is unregistered, the manager is notified and it tells the container to shut down the endpoint.  Those hard coded properties for restRelativeURI and soapRelativeURI would normally be part of the blueprint properties and so settable at runtime triggering a reload.  That in turn triggers the manager to tear down the old endpoint and start up a new one.  Since the MicroWebserviceRegistration can accept the same sort of interceptors and providers that the manager uses I've used that as a way to register error/exceptionhandlers as well and in one case to override the JSON provider configuration.  One department wanted badgerfish and another wanted the root stripped and unwrapped.  Just specifying those differently in the sending bundle I could override the defaults.  If I wanted to I could also use different registrations to expose the same service on different relativeURIs or ports with different configuration information.  Here's an example of a service adding in an exception handler.

                <property name="addProvider">
                            <bean class="com.foo.internal.exceptionhandlers.MyExceptionHandler" />
                  </property

If that's added to the manager's configuration then all endpoints that register with it get that exception handler.  If it is registered with the bundle's MicroWebserviceRegistration then that only applies to it.  In this case it is in an internal package but gets provided via the web service.  The MicroWebserviceRegistration also provides a UUID so when it the bundle is uninstalled and the manager gets notified of the fact it knows which container to pull from its registry and shut down.

Just looking at that EchoService if I have three different divisions with different security and REST/SOAP requirements I can have three managers running via a simple blueprint set up of their service managers with service property filters for company A, company B, company C and when I register the EchoService I can specify different relative URIs but let each of the managers determine how the interface is actually going to be exposed.

It works very well.  In practice I have most of the properties relevant to the bundle's requirements exposed in the cfg files making changes simple.

I'll take a look at RSA tomorrow and see how it works.

Brad

On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider <[hidden email]> wrote:
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com



-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
I took a look at RSA and DOSGi again.  I'd forgotten the RSA acronym so didn't recognize that I had looked at it before.  My conclusion is very similar to your own.  They are orthogonal concerns with a small amount of overlap..  I think what would make the determination of which one to use, for me, would be a very explicit context.  Obviously what I made isn't really about remote management or distributed architecture.  It is more tactical than that and related to specific bundles.  So if someone were to ask me about a solution for a server farm the MicroWebservice would NOT be on the candidate list. The MicroWebservice does use the OSGi whiteboard to listen for the MicroWebservice registration events. Because it always uses the same OSGi registration service interface 1 .. N MicroWebservers can listen for the same events with their list-listeners and you could filter on service properties.  I forget the LDAP filtering syntax off the top of my head.  

I just got tired of having to set up and re-set up so many different blueprint configurations fro different endpoints.  And if you try to centralize that in something like gateway bundle now you have an issue that when the provider bundle is undeployed the web service is still hanging out there in space disconnected.  With this mechanic if you undeploy your bundle all the standard Aries blueprint mechanics trigger the removal of the web services associated with a given bundle.  But all of that relies on your Aries OSGi mechanisms with blueprint to function.  It can't exist without it.  That's part of why I thought of mentioning it here first as it might actually fill a niche need for Aries or Aries users.  If folks here got interested in it then maybe you'd adopt it some day.  Who knows.  But at least I'd get more eyes and thoughts and solidification to it as well.  And if folks don't find it fits any need for them, then I won't waste a lot of time on it either.

For better or worse microservices are the fashion today.  As a consultant I don't get to make the call and don't even bother fighting the trends or fads unless they make specifically bad fits for specific situations.


It's sort of like the library you have that I was looking at a little last night that bootstraps blueprint files without OSGi.  That is simply another way of doing something that Spring, Guice, etc do it but it doesn't make OSGi or Spring obsolete.  But it is nice to have in the toolbox.  I've been using blueprint for the past 4 years or so and being able to use a modified form of it for wiring up and booting little applications is nice.  I don't have to switch to a different dialect like Spring (although I used to use that heavily in the olden days).  So the MicroWebservices is just another way to use the standard CXF, Blueprint and OSGi mechanism to ease the pain of web service set up and configuration.

As I get more, uhm, mature it is fun to watch how new fads come into being.  Docker and Kubernettes are the rage and I certainly understand the underlying technologies.  But how often do you see folks just grabbing hold of them because they are the latest, coolest thing?  I like Docker conceptually but working in karaf/blueprint/OSGi I'm not sure I have that many instances in my enterprise applications where I'd have a great need for them.  On the other hand, if I have a client insistent on them then seeing a tool like your blueprint without OSGi becomes far more interesting.  


On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider <[hidden email]> wrote:
Thanks for the explanations. I think I now understand better.

In effect it does a similar thing but with a different focus. Aries RSA and the CXF provider focus on the whiteboard pattern to kind of transparently export/import a service. Both do not have  strong out of the box support for managing and centralizing configurations. It can be done but it less prepared than in your framework.
This is an example of how to export a rest service:
https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

Your framework is more similar to the CXF blueprint namespace as you have to explicitly export a REST or SOAP service using blueprint. It focuses on the configuration aspect by managing a system wide and per service configuration. Something l really like there is the do a lot of the config system wide this is something I would like to add for Aries RSA too.

I really like the whiteboard approach of RSA as it allows to have no or minimal dependencies from the user code to the framework. So maybe you are interested in combining the capabilities of both.
RSA now has a custom service provider interface ExportPolicy. This is a hook where you can do system wide configuration and also things like auto exporting services based on annotations.

I think you can achieve the same level of manageability in RSA as with your framework but it is not there already. If you are interested in helping out there I can help you to get into the code.

Christian


On 17.04.2016 01:08, Brad Johnson wrote:
I'll have to look at the Aries RSA as I'm not as aware of it.  It doesn't appear to be similar to DOSGi from my experience with it.  I'll take a look at them again tomorrow.  Like so many things in our OSGi, ServiceMix, Camel world there are about 10 different ways to do everything.  Some are better than others and most are better in certain circumstances.  This design is in reaction to consulting on a job that involves a company which is integrating multiple smaller companies that it owns and they all have a little different security, different ports, different web service requirements but are commonly going to be talking to the same Fuse instance in many (but not all) cases.


 This uses the OSGi mechanics for registration only and it doesn't require that the the interface itself be an OSGi service.  Here is a listener on a blueprint file for the MicroWebserviceManager.  I'm not filtering on any service property but but I certainly could.  The manager has a configuration associated with it for standard interceptors, etc. 


<reference-list id="microserviceListener" interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<reference-listener bind-method="register" unbind-method="unregister">
<ref component-id="microserviceManager" />
</reference-listener>
</reference-list>


The simple Echo bundle I have in there has this in its blueprint.

<service interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<bean class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
<argument value="org.enjekt.osgi.api.echo.EchoService" />
<argument ref="impl" />
<property name="restRelativeURI" value="/resources/echo" />
<property name="soapRelativeURI" value="/services/echo" />
                      
</bean>
</service>

That MicroWebServiceRegistration inherits from the same class that the .MicroWebserviceManager uses for configuration.  When this simple Echo bundle starts it registers as a MicroWebService and the manager picks it up and combines the configuration found in both.  As you can see registration isn't specifying the host/port information so the one used by the MicroWebserviceManager will be used.  However, if I did specify a different host/port in the bundle's properties that would be used instead.  The manager itself never actually creates the server/endpoints.  It instantiates a container that it keeps in a map and that container instantiates the server/endpoints based on the combined information.  

Obviously if I uninstall the Echo bundle this particular MicroWebservice is unregistered, the manager is notified and it tells the container to shut down the endpoint.  Those hard coded properties for restRelativeURI and soapRelativeURI would normally be part of the blueprint properties and so settable at runtime triggering a reload.  That in turn triggers the manager to tear down the old endpoint and start up a new one.  Since the MicroWebserviceRegistration can accept the same sort of interceptors and providers that the manager uses I've used that as a way to register error/exceptionhandlers as well and in one case to override the JSON provider configuration.  One department wanted badgerfish and another wanted the root stripped and unwrapped.  Just specifying those differently in the sending bundle I could override the defaults.  If I wanted to I could also use different registrations to expose the same service on different relativeURIs or ports with different configuration information.  Here's an example of a service adding in an exception handler.

                <property name="addProvider">
                            <bean class="com.foo.internal.exceptionhandlers.MyExceptionHandler" />
                  </property

If that's added to the manager's configuration then all endpoints that register with it get that exception handler.  If it is registered with the bundle's MicroWebserviceRegistration then that only applies to it.  In this case it is in an internal package but gets provided via the web service.  The MicroWebserviceRegistration also provides a UUID so when it the bundle is uninstalled and the manager gets notified of the fact it knows which container to pull from its registry and shut down.

Just looking at that EchoService if I have three different divisions with different security and REST/SOAP requirements I can have three managers running via a simple blueprint set up of their service managers with service property filters for company A, company B, company C and when I register the EchoService I can specify different relative URIs but let each of the managers determine how the interface is actually going to be exposed.

It works very well.  In practice I have most of the properties relevant to the bundle's requirements exposed in the cfg files making changes simple.

I'll take a look at RSA tomorrow and see how it works.

Brad

On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider <[hidden email][hidden email]> wrote:
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email][hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com



-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Christian Schneider
I think the main feature of the MicroWebServer approach you show is that you can define multiple endpoints for the same impl. This is indeed a niche that Aries RSA does not fill and does not try to fill. So it might be a good case for people to use it.

About blueprint without OSGi.. I think it is great for testing but if your application really is not OSGi based I would not bother with it and just use spring or another non OSGi dependency injection.

Microservices and docker are indeed very popular at the moment. Still I think it makes sense to use OSGi. On the other hand it would also be overkill to use a full blown karaf server for it as you will only host one application per process. 

We are currently working on karaf boot which attempts to support this style very well. It will support a mainly annotation based developement model like spring boot. On the packaging side you can use the static karaf profile to package your application using karaf features into a fixed set of bundles at build time. So at runtime you still have karaf but without the feature service so the runtime is simpler and nicely suited to host one single application.

Christian

2016-04-17 17:14 GMT+02:00 Brad Johnson <[hidden email]>:
I took a look at RSA and DOSGi again.  I'd forgotten the RSA acronym so didn't recognize that I had looked at it before.  My conclusion is very similar to your own.  They are orthogonal concerns with a small amount of overlap..  I think what would make the determination of which one to use, for me, would be a very explicit context.  Obviously what I made isn't really about remote management or distributed architecture.  It is more tactical than that and related to specific bundles.  So if someone were to ask me about a solution for a server farm the MicroWebservice would NOT be on the candidate list. The MicroWebservice does use the OSGi whiteboard to listen for the MicroWebservice registration events. Because it always uses the same OSGi registration service interface 1 .. N MicroWebservers can listen for the same events with their list-listeners and you could filter on service properties.  I forget the LDAP filtering syntax off the top of my head.  

I just got tired of having to set up and re-set up so many different blueprint configurations fro different endpoints.  And if you try to centralize that in something like gateway bundle now you have an issue that when the provider bundle is undeployed the web service is still hanging out there in space disconnected.  With this mechanic if you undeploy your bundle all the standard Aries blueprint mechanics trigger the removal of the web services associated with a given bundle.  But all of that relies on your Aries OSGi mechanisms with blueprint to function.  It can't exist without it.  That's part of why I thought of mentioning it here first as it might actually fill a niche need for Aries or Aries users.  If folks here got interested in it then maybe you'd adopt it some day.  Who knows.  But at least I'd get more eyes and thoughts and solidification to it as well.  And if folks don't find it fits any need for them, then I won't waste a lot of time on it either.

For better or worse microservices are the fashion today.  As a consultant I don't get to make the call and don't even bother fighting the trends or fads unless they make specifically bad fits for specific situations.


It's sort of like the library you have that I was looking at a little last night that bootstraps blueprint files without OSGi.  That is simply another way of doing something that Spring, Guice, etc do it but it doesn't make OSGi or Spring obsolete.  But it is nice to have in the toolbox.  I've been using blueprint for the past 4 years or so and being able to use a modified form of it for wiring up and booting little applications is nice.  I don't have to switch to a different dialect like Spring (although I used to use that heavily in the olden days).  So the MicroWebservices is just another way to use the standard CXF, Blueprint and OSGi mechanism to ease the pain of web service set up and configuration.

As I get more, uhm, mature it is fun to watch how new fads come into being.  Docker and Kubernettes are the rage and I certainly understand the underlying technologies.  But how often do you see folks just grabbing hold of them because they are the latest, coolest thing?  I like Docker conceptually but working in karaf/blueprint/OSGi I'm not sure I have that many instances in my enterprise applications where I'd have a great need for them.  On the other hand, if I have a client insistent on them then seeing a tool like your blueprint without OSGi becomes far more interesting.  


On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider <[hidden email]> wrote:
Thanks for the explanations. I think I now understand better.

In effect it does a similar thing but with a different focus. Aries RSA and the CXF provider focus on the whiteboard pattern to kind of transparently export/import a service. Both do not have  strong out of the box support for managing and centralizing configurations. It can be done but it less prepared than in your framework.
This is an example of how to export a rest service:
https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

Your framework is more similar to the CXF blueprint namespace as you have to explicitly export a REST or SOAP service using blueprint. It focuses on the configuration aspect by managing a system wide and per service configuration. Something l really like there is the do a lot of the config system wide this is something I would like to add for Aries RSA too.

I really like the whiteboard approach of RSA as it allows to have no or minimal dependencies from the user code to the framework. So maybe you are interested in combining the capabilities of both.
RSA now has a custom service provider interface ExportPolicy. This is a hook where you can do system wide configuration and also things like auto exporting services based on annotations.

I think you can achieve the same level of manageability in RSA as with your framework but it is not there already. If you are interested in helping out there I can help you to get into the code.

Christian


On 17.04.2016 01:08, Brad Johnson wrote:
I'll have to look at the Aries RSA as I'm not as aware of it.  It doesn't appear to be similar to DOSGi from my experience with it.  I'll take a look at them again tomorrow.  Like so many things in our OSGi, ServiceMix, Camel world there are about 10 different ways to do everything.  Some are better than others and most are better in certain circumstances.  This design is in reaction to consulting on a job that involves a company which is integrating multiple smaller companies that it owns and they all have a little different security, different ports, different web service requirements but are commonly going to be talking to the same Fuse instance in many (but not all) cases.


 This uses the OSGi mechanics for registration only and it doesn't require that the the interface itself be an OSGi service.  Here is a listener on a blueprint file for the MicroWebserviceManager.  I'm not filtering on any service property but but I certainly could.  The manager has a configuration associated with it for standard interceptors, etc. 


<reference-list id="microserviceListener" interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<reference-listener bind-method="register" unbind-method="unregister">
<ref component-id="microserviceManager" />
</reference-listener>
</reference-list>


The simple Echo bundle I have in there has this in its blueprint.

<service interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<bean class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
<argument value="org.enjekt.osgi.api.echo.EchoService" />
<argument ref="impl" />
<property name="restRelativeURI" value="/resources/echo" />
<property name="soapRelativeURI" value="/services/echo" />
                      
</bean>
</service>

That MicroWebServiceRegistration inherits from the same class that the .MicroWebserviceManager uses for configuration.  When this simple Echo bundle starts it registers as a MicroWebService and the manager picks it up and combines the configuration found in both.  As you can see registration isn't specifying the host/port information so the one used by the MicroWebserviceManager will be used.  However, if I did specify a different host/port in the bundle's properties that would be used instead.  The manager itself never actually creates the server/endpoints.  It instantiates a container that it keeps in a map and that container instantiates the server/endpoints based on the combined information.  

Obviously if I uninstall the Echo bundle this particular MicroWebservice is unregistered, the manager is notified and it tells the container to shut down the endpoint.  Those hard coded properties for restRelativeURI and soapRelativeURI would normally be part of the blueprint properties and so settable at runtime triggering a reload.  That in turn triggers the manager to tear down the old endpoint and start up a new one.  Since the MicroWebserviceRegistration can accept the same sort of interceptors and providers that the manager uses I've used that as a way to register error/exceptionhandlers as well and in one case to override the JSON provider configuration.  One department wanted badgerfish and another wanted the root stripped and unwrapped.  Just specifying those differently in the sending bundle I could override the defaults.  If I wanted to I could also use different registrations to expose the same service on different relativeURIs or ports with different configuration information.  Here's an example of a service adding in an exception handler.

                <property name="addProvider">
                            <bean class="com.foo.internal.exceptionhandlers.MyExceptionHandler" />
                  </property

If that's added to the manager's configuration then all endpoints that register with it get that exception handler.  If it is registered with the bundle's MicroWebserviceRegistration then that only applies to it.  In this case it is in an internal package but gets provided via the web service.  The MicroWebserviceRegistration also provides a UUID so when it the bundle is uninstalled and the manager gets notified of the fact it knows which container to pull from its registry and shut down.

Just looking at that EchoService if I have three different divisions with different security and REST/SOAP requirements I can have three managers running via a simple blueprint set up of their service managers with service property filters for company A, company B, company C and when I register the EchoService I can specify different relative URIs but let each of the managers determine how the interface is actually going to be exposed.

It works very well.  In practice I have most of the properties relevant to the bundle's requirements exposed in the cfg files making changes simple.

I'll take a look at RSA tomorrow and see how it works.

Brad

On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider <[hidden email][hidden email]> wrote:
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email][hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com



-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com




--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
I think you hit it with your observation about why even use that MicroWebServer component.  Different requirements on different addresses and customizations.  Also the bundle itself can come and go and have its web services deploy and undeploy without having to define server endpoints in the bundle itself.  If my EchoService bundle is installed and there isn't MicroWebServer installed it simply installs as another OSGi bundle whose local service interface might be used by other bundles.  But it isn't itself defining any CXF server information that may or may not be wanted at deployment time.  It separates the two concerns a bit.

I'm not in the rush or on the bandwagon to adopt Docker but one does have to be prepared.  The one thing I'd like about having a NoOSGi version of blueprint is not having to jump between different XML bootstrap mechanics - Spring or Guice then back to Blueprint.  Certainly Spring is close enough to Blueprint that the syntax is easy enough but now one has different mechanics all the way from XML up through builds.  Not a huge deal but if I can make small applications with Blueprint that just eases life a little.  I'm in an environment right now where Fuse is the central focal point so the OSGi, Camel, Blueprint are not going away anytime soon. But I'm also getting where other departments want us to develop and deploy small self-contained services for them.  As you point out, even a minimal karaf/servicemix implementation can be big for that.  I also have the irritant of dealing with licensing issues because of the corporate environment.  I can't just plop minimal installs of Fuse in different divisions because of licensing.  I say it's irritating only because that means I have to fish for technical solutions that aren't addressing a technical or business issue but a legal one.

But initially it will be for testing as being able to set up stub services for code to call out to will be quite handy.

On Sun, Apr 17, 2016 at 10:42 AM, Christian Schneider <[hidden email]> wrote:
I think the main feature of the MicroWebServer approach you show is that you can define multiple endpoints for the same impl. This is indeed a niche that Aries RSA does not fill and does not try to fill. So it might be a good case for people to use it.

About blueprint without OSGi.. I think it is great for testing but if your application really is not OSGi based I would not bother with it and just use spring or another non OSGi dependency injection.

Microservices and docker are indeed very popular at the moment. Still I think it makes sense to use OSGi. On the other hand it would also be overkill to use a full blown karaf server for it as you will only host one application per process. 

We are currently working on karaf boot which attempts to support this style very well. It will support a mainly annotation based developement model like spring boot. On the packaging side you can use the static karaf profile to package your application using karaf features into a fixed set of bundles at build time. So at runtime you still have karaf but without the feature service so the runtime is simpler and nicely suited to host one single application.

Christian

2016-04-17 17:14 GMT+02:00 Brad Johnson <[hidden email]>:
I took a look at RSA and DOSGi again.  I'd forgotten the RSA acronym so didn't recognize that I had looked at it before.  My conclusion is very similar to your own.  They are orthogonal concerns with a small amount of overlap..  I think what would make the determination of which one to use, for me, would be a very explicit context.  Obviously what I made isn't really about remote management or distributed architecture.  It is more tactical than that and related to specific bundles.  So if someone were to ask me about a solution for a server farm the MicroWebservice would NOT be on the candidate list. The MicroWebservice does use the OSGi whiteboard to listen for the MicroWebservice registration events. Because it always uses the same OSGi registration service interface 1 .. N MicroWebservers can listen for the same events with their list-listeners and you could filter on service properties.  I forget the LDAP filtering syntax off the top of my head.  

I just got tired of having to set up and re-set up so many different blueprint configurations fro different endpoints.  And if you try to centralize that in something like gateway bundle now you have an issue that when the provider bundle is undeployed the web service is still hanging out there in space disconnected.  With this mechanic if you undeploy your bundle all the standard Aries blueprint mechanics trigger the removal of the web services associated with a given bundle.  But all of that relies on your Aries OSGi mechanisms with blueprint to function.  It can't exist without it.  That's part of why I thought of mentioning it here first as it might actually fill a niche need for Aries or Aries users.  If folks here got interested in it then maybe you'd adopt it some day.  Who knows.  But at least I'd get more eyes and thoughts and solidification to it as well.  And if folks don't find it fits any need for them, then I won't waste a lot of time on it either.

For better or worse microservices are the fashion today.  As a consultant I don't get to make the call and don't even bother fighting the trends or fads unless they make specifically bad fits for specific situations.


It's sort of like the library you have that I was looking at a little last night that bootstraps blueprint files without OSGi.  That is simply another way of doing something that Spring, Guice, etc do it but it doesn't make OSGi or Spring obsolete.  But it is nice to have in the toolbox.  I've been using blueprint for the past 4 years or so and being able to use a modified form of it for wiring up and booting little applications is nice.  I don't have to switch to a different dialect like Spring (although I used to use that heavily in the olden days).  So the MicroWebservices is just another way to use the standard CXF, Blueprint and OSGi mechanism to ease the pain of web service set up and configuration.

As I get more, uhm, mature it is fun to watch how new fads come into being.  Docker and Kubernettes are the rage and I certainly understand the underlying technologies.  But how often do you see folks just grabbing hold of them because they are the latest, coolest thing?  I like Docker conceptually but working in karaf/blueprint/OSGi I'm not sure I have that many instances in my enterprise applications where I'd have a great need for them.  On the other hand, if I have a client insistent on them then seeing a tool like your blueprint without OSGi becomes far more interesting.  


On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider <[hidden email]> wrote:
Thanks for the explanations. I think I now understand better.

In effect it does a similar thing but with a different focus. Aries RSA and the CXF provider focus on the whiteboard pattern to kind of transparently export/import a service. Both do not have  strong out of the box support for managing and centralizing configurations. It can be done but it less prepared than in your framework.
This is an example of how to export a rest service:
https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

Your framework is more similar to the CXF blueprint namespace as you have to explicitly export a REST or SOAP service using blueprint. It focuses on the configuration aspect by managing a system wide and per service configuration. Something l really like there is the do a lot of the config system wide this is something I would like to add for Aries RSA too.

I really like the whiteboard approach of RSA as it allows to have no or minimal dependencies from the user code to the framework. So maybe you are interested in combining the capabilities of both.
RSA now has a custom service provider interface ExportPolicy. This is a hook where you can do system wide configuration and also things like auto exporting services based on annotations.

I think you can achieve the same level of manageability in RSA as with your framework but it is not there already. If you are interested in helping out there I can help you to get into the code.

Christian


On 17.04.2016 01:08, Brad Johnson wrote:
I'll have to look at the Aries RSA as I'm not as aware of it.  It doesn't appear to be similar to DOSGi from my experience with it.  I'll take a look at them again tomorrow.  Like so many things in our OSGi, ServiceMix, Camel world there are about 10 different ways to do everything.  Some are better than others and most are better in certain circumstances.  This design is in reaction to consulting on a job that involves a company which is integrating multiple smaller companies that it owns and they all have a little different security, different ports, different web service requirements but are commonly going to be talking to the same Fuse instance in many (but not all) cases.


 This uses the OSGi mechanics for registration only and it doesn't require that the the interface itself be an OSGi service.  Here is a listener on a blueprint file for the MicroWebserviceManager.  I'm not filtering on any service property but but I certainly could.  The manager has a configuration associated with it for standard interceptors, etc. 


<reference-list id="microserviceListener" interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<reference-listener bind-method="register" unbind-method="unregister">
<ref component-id="microserviceManager" />
</reference-listener>
</reference-list>


The simple Echo bundle I have in there has this in its blueprint.

<service interface="org.enjekt.osgi.microserver.api.MicroWebservice">
<bean class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
<argument value="org.enjekt.osgi.api.echo.EchoService" />
<argument ref="impl" />
<property name="restRelativeURI" value="/resources/echo" />
<property name="soapRelativeURI" value="/services/echo" />
                      
</bean>
</service>

That MicroWebServiceRegistration inherits from the same class that the .MicroWebserviceManager uses for configuration.  When this simple Echo bundle starts it registers as a MicroWebService and the manager picks it up and combines the configuration found in both.  As you can see registration isn't specifying the host/port information so the one used by the MicroWebserviceManager will be used.  However, if I did specify a different host/port in the bundle's properties that would be used instead.  The manager itself never actually creates the server/endpoints.  It instantiates a container that it keeps in a map and that container instantiates the server/endpoints based on the combined information.  

Obviously if I uninstall the Echo bundle this particular MicroWebservice is unregistered, the manager is notified and it tells the container to shut down the endpoint.  Those hard coded properties for restRelativeURI and soapRelativeURI would normally be part of the blueprint properties and so settable at runtime triggering a reload.  That in turn triggers the manager to tear down the old endpoint and start up a new one.  Since the MicroWebserviceRegistration can accept the same sort of interceptors and providers that the manager uses I've used that as a way to register error/exceptionhandlers as well and in one case to override the JSON provider configuration.  One department wanted badgerfish and another wanted the root stripped and unwrapped.  Just specifying those differently in the sending bundle I could override the defaults.  If I wanted to I could also use different registrations to expose the same service on different relativeURIs or ports with different configuration information.  Here's an example of a service adding in an exception handler.

                <property name="addProvider">
                            <bean class="com.foo.internal.exceptionhandlers.MyExceptionHandler" />
                  </property

If that's added to the manager's configuration then all endpoints that register with it get that exception handler.  If it is registered with the bundle's MicroWebserviceRegistration then that only applies to it.  In this case it is in an internal package but gets provided via the web service.  The MicroWebserviceRegistration also provides a UUID so when it the bundle is uninstalled and the manager gets notified of the fact it knows which container to pull from its registry and shut down.

Just looking at that EchoService if I have three different divisions with different security and REST/SOAP requirements I can have three managers running via a simple blueprint set up of their service managers with service property filters for company A, company B, company C and when I register the EchoService I can specify different relative URIs but let each of the managers determine how the interface is actually going to be exposed.

It works very well.  In practice I have most of the properties relevant to the bundle's requirements exposed in the cfg files making changes simple.

I'll take a look at RSA tomorrow and see how it works.

Brad

On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider <[hidden email][hidden email]> wrote:
Hi Brad,

from what I understood it allows to expose OSGi services as SOAP and REST endpoints. This sounds very similar to Aries RSA and CXF DOSGi. Can you explain what it does differently?

Christian

2016-04-16 17:39 GMT+02:00 Brad Johnson <[hidden email][hidden email]>:
I've started a project that may or may not be of interest to the community.  I've been finding it of great benefit in my current projects where I combine it with Camel for certain items.  One change I plan on making as soon as I can get to it is to change the BOM from importing Fuse dependencies to Servicemix.  That should be relatively simple but I'm working with it in Fuse currently so it hasn't been a high priority.

Any feedback is appreciated.

Brad



--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com



-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com




--
--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Jean-Baptiste Onofré
Hi Brad,

I second Christian. I think that leveraging OSGi services is very
interesting to build non-monolithic microservices (largely better than
big monolithic approach).

As Christian said, karaf-boot should help a lot there as it will:

1. Simplify the way of building application thanks to annotation
2. Create distribution as single jar, profile, dockerfile, docker image,
etc. It's the "run everywhere" paradign. Right now, you have first to
install Karaf and then deploy your application in Karaf. With
karaf-boot, you focus on your business code, and karaf-boot deals with
the rest (up to the distribution).

Regards
JB

On 04/17/2016 06:15 PM, Brad Johnson wrote:

> I think you hit it with your observation about why even use that
> MicroWebServer component.  Different requirements on different addresses
> and customizations.  Also the bundle itself can come and go and have its
> web services deploy and undeploy without having to define server
> endpoints in the bundle itself.  If my EchoService bundle is installed
> and there isn't MicroWebServer installed it simply installs as another
> OSGi bundle whose local service interface might be used by other
> bundles.  But it isn't itself defining any CXF server information that
> may or may not be wanted at deployment time.  It separates the two
> concerns a bit.
>
> I'm not in the rush or on the bandwagon to adopt Docker but one does
> have to be prepared.  The one thing I'd like about having a NoOSGi
> version of blueprint is not having to jump between different XML
> bootstrap mechanics - Spring or Guice then back to Blueprint.  Certainly
> Spring is close enough to Blueprint that the syntax is easy enough but
> now one has different mechanics all the way from XML up through builds.
> Not a huge deal but if I can make small applications with Blueprint that
> just eases life a little.  I'm in an environment right now where Fuse is
> the central focal point so the OSGi, Camel, Blueprint are not going away
> anytime soon. But I'm also getting where other departments want us to
> develop and deploy small self-contained services for them.  As you point
> out, even a minimal karaf/servicemix implementation can be big for
> that.  I also have the irritant of dealing with licensing issues because
> of the corporate environment.  I can't just plop minimal installs of
> Fuse in different divisions because of licensing.  I say it's irritating
> only because that means I have to fish for technical solutions that
> aren't addressing a technical or business issue but a legal one.
>
> But initially it will be for testing as being able to set up stub
> services for code to call out to will be quite handy.
>
> On Sun, Apr 17, 2016 at 10:42 AM, Christian Schneider
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     I think the main feature of the MicroWebServer approach you show is
>     that you can define multiple endpoints for the same impl. This is
>     indeed a niche that Aries RSA does not fill and does not try to
>     fill. So it might be a good case for people to use it.
>
>     About blueprint without OSGi.. I think it is great for testing but
>     if your application really is not OSGi based I would not bother with
>     it and just use spring or another non OSGi dependency injection.
>
>     Microservices and docker are indeed very popular at the moment.
>     Still I think it makes sense to use OSGi. On the other hand it would
>     also be overkill to use a full blown karaf server for it as you will
>     only host one application per process.
>
>     We are currently working on karaf boot which attempts to support
>     this style very well. It will support a mainly annotation based
>     developement model like spring boot. On the packaging side you can
>     use the static karaf profile to package your application using karaf
>     features into a fixed set of bundles at build time. So at runtime
>     you still have karaf but without the feature service so the runtime
>     is simpler and nicely suited to host one single application.
>
>     Christian
>
>     2016-04-17 17:14 GMT+02:00 Brad Johnson
>     <[hidden email] <mailto:[hidden email]>>:
>
>         I took a look at RSA and DOSGi again.  I'd forgotten the RSA
>         acronym so didn't recognize that I had looked at it before.  My
>         conclusion is very similar to your own.  They are orthogonal
>         concerns with a small amount of overlap..  I think what would
>         make the determination of which one to use, for me, would be a
>         very explicit context.  Obviously what I made isn't really about
>         remote management or distributed architecture.  It is more
>         tactical than that and related to specific bundles.  So if
>         someone were to ask me about a solution for a server farm the
>         MicroWebservice would NOT be on the candidate list. The
>         MicroWebservice does use the OSGi whiteboard to listen for the
>         MicroWebservice registration events. Because it always uses the
>         same OSGi registration service interface 1 .. N MicroWebservers
>         can listen for the same events with their list-listeners and you
>         could filter on service properties.  I forget the LDAP filtering
>         syntax off the top of my head.
>
>         I just got tired of having to set up and re-set up so many
>         different blueprint configurations fro different endpoints.  And
>         if you try to centralize that in something like gateway bundle
>         now you have an issue that when the provider bundle is
>         undeployed the web service is still hanging out there in space
>         disconnected.  With this mechanic if you undeploy your bundle
>         all the standard Aries blueprint mechanics trigger the removal
>         of the web services associated with a given bundle.  But all of
>         that relies on your Aries OSGi mechanisms with blueprint to
>         function.  It can't exist without it.  That's part of why I
>         thought of mentioning it here first as it might actually fill a
>         niche need for Aries or Aries users.  If folks here got
>         interested in it then maybe you'd adopt it some day.  Who
>         knows.  But at least I'd get more eyes and thoughts and
>         solidification to it as well.  And if folks don't find it fits
>         any need for them, then I won't waste a lot of time on it either.
>
>         For better or worse microservices are the fashion today.  As a
>         consultant I don't get to make the call and don't even bother
>         fighting the trends or fads unless they make specifically bad
>         fits for specific situations.
>
>
>         It's sort of like the library you have that I was looking at a
>         little last night that bootstraps blueprint files without OSGi.
>         That is simply another way of doing something that Spring,
>         Guice, etc do it but it doesn't make OSGi or Spring obsolete.
>         But it is nice to have in the toolbox.  I've been using
>         blueprint for the past 4 years or so and being able to use a
>         modified form of it for wiring up and booting little
>         applications is nice.  I don't have to switch to a different
>         dialect like Spring (although I used to use that heavily in the
>         olden days).  So the MicroWebservices is just another way to use
>         the standard CXF, Blueprint and OSGi mechanism to ease the pain
>         of web service set up and configuration.
>
>         As I get more, uhm, mature it is fun to watch how new fads come
>         into being.  Docker and Kubernettes are the rage and I certainly
>         understand the underlying technologies.  But how often do you
>         see folks just grabbing hold of them because they are the
>         latest, coolest thing?  I like Docker conceptually but working
>         in karaf/blueprint/OSGi I'm not sure I have that many instances
>         in my enterprise applications where I'd have a great need for
>         them.  On the other hand, if I have a client insistent on them
>         then seeing a tool like your blueprint without OSGi becomes far
>         more interesting.
>
>
>         On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider
>         <[hidden email] <mailto:[hidden email]>> wrote:
>
>             Thanks for the explanations. I think I now understand better.
>
>             In effect it does a similar thing but with a different
>             focus. Aries RSA and the CXF provider focus on the
>             whiteboard pattern to kind of transparently export/import a
>             service. Both do not have strong out of the box support for
>             managing and centralizing configurations. It can be done but
>             it less prepared than in your framework.
>             This is an example of how to export a rest service:
>             https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java
>
>             Your framework is more similar to the CXF blueprint
>             namespace as you have to explicitly export a REST or SOAP
>             service using blueprint. It focuses on the configuration
>             aspect by managing a system wide and per service
>             configuration. Something l really like there is the do a lot
>             of the config system wide this is something I would like to
>             add for Aries RSA too.
>
>             I really like the whiteboard approach of RSA as it allows to
>             have no or minimal dependencies from the user code to the
>             framework. So maybe you are interested in combining the
>             capabilities of both.
>             RSA now has a custom service provider interface
>             ExportPolicy. This is a hook where you can do system wide
>             configuration and also things like auto exporting services
>             based on annotations.
>
>             I think you can achieve the same level of manageability in
>             RSA as with your framework but it is not there already. If
>             you are interested in helping out there I can help you to
>             get into the code.
>
>             Christian
>
>
>             On 17.04.2016 01:08, Brad Johnson wrote:
>>             I'll have to look at the Aries RSA as I'm not as aware of
>>             it.  It doesn't appear to be similar to DOSGi from my
>>             experience with it.  I'll take a look at them again
>>             tomorrow. Like so many things in our OSGi, ServiceMix,
>>             Camel world there are about 10 different ways to do
>>             everything.  Some are better than others and most are
>>             better in certain circumstances.  This design is in
>>             reaction to consulting on a job that involves a company
>>             which is integrating multiple smaller companies that it
>>             owns and they all have a little different security,
>>             different ports, different web service requirements but
>>             are commonly going to be talking to the same Fuse instance
>>             in many (but not all) cases.
>>
>>
>>              This uses the OSGi mechanics for registration only and it
>>             doesn't require that the the interface itself be an OSGi
>>             service.  Here is a listener on a blueprint file for the
>>             MicroWebserviceManager.  I'm not filtering on any service
>>             property but but I certainly could.  The manager has a
>>             configuration associated with it for standard
>>             interceptors, etc.
>>
>>
>>             <reference-list id="microserviceListener"
>>             interface="org.enjekt.osgi.microserver.api.MicroWebservice">
>>             <reference-listener bind-method="register"
>>             unbind-method="unregister">
>>             <ref component-id="microserviceManager" />
>>             </reference-listener>
>>             </reference-list>
>>
>>
>>             The simple Echo bundle I have in there has this in its
>>             blueprint.
>>
>>             <service
>>             interface="org.enjekt.osgi.microserver.api.MicroWebservice">
>>             <bean
>>             class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
>>             <argument value="org.enjekt.osgi.api.echo.EchoService" />
>>             <argument ref="impl" />
>>             <property name="restRelativeURI" value="/resources/echo" />
>>             <property name="soapRelativeURI" value="/services/echo" />
>>
>>             </bean>
>>             </service>
>>
>>             That MicroWebServiceRegistration inherits from the same
>>             class that the .MicroWebserviceManager uses for
>>             configuration.  When this simple Echo bundle starts it
>>             registers as a MicroWebService and the manager picks it up
>>             and combines the configuration found in both.  As you can
>>             see registration isn't specifying the host/port
>>             information so the one used by the MicroWebserviceManager
>>             will be used.  However, if I did specify a different
>>             host/port in the bundle's properties that would be used
>>             instead.  The manager itself never actually creates the
>>             server/endpoints.  It instantiates a container that it
>>             keeps in a map and that container instantiates the
>>             server/endpoints based on the combined information.
>>
>>             Obviously if I uninstall the Echo bundle this particular
>>             MicroWebservice is unregistered, the manager is notified
>>             and it tells the container to shut down the endpoint.
>>             Those hard coded properties for restRelativeURI and
>>             soapRelativeURI would normally be part of the blueprint
>>             properties and so settable at runtime triggering a
>>             reload.  That in turn triggers the manager to tear down
>>             the old endpoint and start up a new one. Since the
>>             MicroWebserviceRegistration can accept the same sort of
>>             interceptors and providers that the manager uses I've used
>>             that as a way to register error/exceptionhandlers as well
>>             and in one case to override the JSON provider
>>             configuration.  One department wanted badgerfish and
>>             another wanted the root stripped and unwrapped.  Just
>>             specifying those differently in the sending bundle I could
>>             override the defaults.  If I wanted to I could also use
>>             different registrations to expose the same service on
>>             different relativeURIs or ports with different
>>             configuration information.  Here's an example of a service
>>             adding in an exception handler.
>>
>>                             <property name="addProvider">
>>                                         <bean
>>             class="com.foo.internal.exceptionhandlers.MyExceptionHandler"
>>             />
>>                               </property
>>
>>             If that's added to the manager's configuration then all
>>             endpoints that register with it get that exception
>>             handler. If it is registered with the bundle's
>>             MicroWebserviceRegistration then that only applies to it.
>>             In this case it is in an internal package but gets
>>             provided via the web service.  The
>>             MicroWebserviceRegistration also provides a UUID so when
>>             it the bundle is uninstalled and the manager gets notified
>>             of the fact it knows which container to pull from its
>>             registry and shut down.
>>
>>             Just looking at that EchoService if I have three different
>>             divisions with different security and REST/SOAP
>>             requirements I can have three managers running via a
>>             simple blueprint set up of their service managers with
>>             service property filters for company A, company B, company
>>             C and when I register the EchoService I can specify
>>             different relative URIs but let each of the managers
>>             determine how the interface is actually going to be exposed.
>>
>>             It works very well.  In practice I have most of the
>>             properties relevant to the bundle's requirements exposed
>>             in the cfg files making changes simple.
>>
>>             I'll take a look at RSA tomorrow and see how it works.
>>
>>             Brad
>>
>>             On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider
>>             <<mailto:[hidden email]>[hidden email]
>>             <mailto:[hidden email]>> wrote:
>>
>>                 Hi Brad,
>>
>>                 from what I understood it allows to expose OSGi
>>                 services as SOAP and REST endpoints. This sounds very
>>                 similar to Aries RSA and CXF DOSGi. Can you explain
>>                 what it does differently?
>>
>>                 Christian
>>
>>                 2016-04-16 17:39 GMT+02:00 Brad Johnson
>>                 <<mailto:[hidden email]>[hidden email]
>>                 <mailto:[hidden email]>>:
>>
>>                     https://github.com/Ranx0r0x/Enjekt-Microservice
>>
>>                     I've started a project that may or may not be of
>>                     interest to the community.  I've been finding it
>>                     of great benefit in my current projects where I
>>                     combine it with Camel for certain items.  One
>>                     change I plan on making as soon as I can get to it
>>                     is to change the BOM from importing Fuse
>>                     dependencies to Servicemix.  That should be
>>                     relatively simple but I'm working with it in Fuse
>>                     currently so it hasn't been a high priority.
>>
>>                     Any feedback is appreciated.
>>
>>                     Brad
>>
>>
>>
>>
>>                 --
>>                 --
>>                 Christian Schneider
>>                 http://www.liquid-reality.de
>>                 <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
>>
>>                 Open Source Architect
>>                 http://www.talend.com
>>                 <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>
>>
>>
>
>
>             --
>             Christian Schneider
>             http://www.liquid-reality.de
>
>             Open Source Architect
>             http://www.talend.com
>
>
>
>
>
>     --
>     --
>     Christian Schneider
>     http://www.liquid-reality.de
>     <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
>
>     Open Source Architect
>     http://www.talend.com
>     <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
Sounds like you both have your reservations about using the noOSGi version of blueprint and you obviously know it better than the rest of us.  When do you anticipate something like Karaf boot working?  I think the Karaf site has 7/15 as the target date for Cellar and 6/16 for karaf 4.1. 

https://karaf.apache.org/projects.html#boot

Since boot relies on Cellar I'm guess that some of the work going into those dates are in support of this project.  It looks like a happy day.

The current size of Karaf is about 20MB so it certainly should be slim enough for the sort of enterprise containers that might be used via Karaf boot.  It'll be interesting to see where some of the lines get drawn.  I've read some on Docker and played with it a little but don't know it well enough.  But I could see cases where each image didn't contain its own Karaf but used a container at a more global/OS level in the way Docker shares other parts of the OS or where it might be specified as a Karaf instance wrapped in each image itself.  Guess I'm going to have to get more knowledgeable about it.

I'll display some frank ignorance here.  When I move up out of the OSGi container world I've lived in for the past number of years with Camel and blueprint some of the areas where concerns are competitive and where they are simply orthogonal to one another is blurry.  Kubernetes is an obvious player in this market and I'm sure that if work is on-going with Karaf-boot there must be plans in place for it as well.

As always, our software world is an exciting and confusing world.

Brad

On Sun, Apr 17, 2016 at 2:18 PM, Jean-Baptiste Onofré <[hidden email]> wrote:
Hi Brad,

I second Christian. I think that leveraging OSGi services is very interesting to build non-monolithic microservices (largely better than big monolithic approach).

As Christian said, karaf-boot should help a lot there as it will:

1. Simplify the way of building application thanks to annotation
2. Create distribution as single jar, profile, dockerfile, docker image, etc. It's the "run everywhere" paradign. Right now, you have first to install Karaf and then deploy your application in Karaf. With karaf-boot, you focus on your business code, and karaf-boot deals with the rest (up to the distribution).

Regards
JB


On 04/17/2016 06:15 PM, Brad Johnson wrote:
I think you hit it with your observation about why even use that
MicroWebServer component.  Different requirements on different addresses
and customizations.  Also the bundle itself can come and go and have its
web services deploy and undeploy without having to define server
endpoints in the bundle itself.  If my EchoService bundle is installed
and there isn't MicroWebServer installed it simply installs as another
OSGi bundle whose local service interface might be used by other
bundles.  But it isn't itself defining any CXF server information that
may or may not be wanted at deployment time.  It separates the two
concerns a bit.

I'm not in the rush or on the bandwagon to adopt Docker but one does
have to be prepared.  The one thing I'd like about having a NoOSGi
version of blueprint is not having to jump between different XML
bootstrap mechanics - Spring or Guice then back to Blueprint.  Certainly
Spring is close enough to Blueprint that the syntax is easy enough but
now one has different mechanics all the way from XML up through builds.
Not a huge deal but if I can make small applications with Blueprint that
just eases life a little.  I'm in an environment right now where Fuse is
the central focal point so the OSGi, Camel, Blueprint are not going away
anytime soon. But I'm also getting where other departments want us to
develop and deploy small self-contained services for them.  As you point
out, even a minimal karaf/servicemix implementation can be big for
that.  I also have the irritant of dealing with licensing issues because
of the corporate environment.  I can't just plop minimal installs of
Fuse in different divisions because of licensing.  I say it's irritating
only because that means I have to fish for technical solutions that
aren't addressing a technical or business issue but a legal one.

But initially it will be for testing as being able to set up stub
services for code to call out to will be quite handy.

On Sun, Apr 17, 2016 at 10:42 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I think the main feature of the MicroWebServer approach you show is
    that you can define multiple endpoints for the same impl. This is
    indeed a niche that Aries RSA does not fill and does not try to
    fill. So it might be a good case for people to use it.

    About blueprint without OSGi.. I think it is great for testing but
    if your application really is not OSGi based I would not bother with
    it and just use spring or another non OSGi dependency injection.

    Microservices and docker are indeed very popular at the moment.
    Still I think it makes sense to use OSGi. On the other hand it would
    also be overkill to use a full blown karaf server for it as you will
    only host one application per process.

    We are currently working on karaf boot which attempts to support
    this style very well. It will support a mainly annotation based
    developement model like spring boot. On the packaging side you can
    use the static karaf profile to package your application using karaf
    features into a fixed set of bundles at build time. So at runtime
    you still have karaf but without the feature service so the runtime
    is simpler and nicely suited to host one single application.

    Christian

    2016-04-17 17:14 GMT+02:00 Brad Johnson
    <[hidden email] <mailto:[hidden email]>>:


        I took a look at RSA and DOSGi again.  I'd forgotten the RSA
        acronym so didn't recognize that I had looked at it before.  My
        conclusion is very similar to your own.  They are orthogonal
        concerns with a small amount of overlap..  I think what would
        make the determination of which one to use, for me, would be a
        very explicit context.  Obviously what I made isn't really about
        remote management or distributed architecture.  It is more
        tactical than that and related to specific bundles.  So if
        someone were to ask me about a solution for a server farm the
        MicroWebservice would NOT be on the candidate list. The
        MicroWebservice does use the OSGi whiteboard to listen for the
        MicroWebservice registration events. Because it always uses the
        same OSGi registration service interface 1 .. N MicroWebservers
        can listen for the same events with their list-listeners and you
        could filter on service properties.  I forget the LDAP filtering
        syntax off the top of my head.

        I just got tired of having to set up and re-set up so many
        different blueprint configurations fro different endpoints.  And
        if you try to centralize that in something like gateway bundle
        now you have an issue that when the provider bundle is
        undeployed the web service is still hanging out there in space
        disconnected.  With this mechanic if you undeploy your bundle
        all the standard Aries blueprint mechanics trigger the removal
        of the web services associated with a given bundle.  But all of
        that relies on your Aries OSGi mechanisms with blueprint to
        function.  It can't exist without it.  That's part of why I
        thought of mentioning it here first as it might actually fill a
        niche need for Aries or Aries users.  If folks here got
        interested in it then maybe you'd adopt it some day.  Who
        knows.  But at least I'd get more eyes and thoughts and
        solidification to it as well.  And if folks don't find it fits
        any need for them, then I won't waste a lot of time on it either.

        For better or worse microservices are the fashion today.  As a
        consultant I don't get to make the call and don't even bother
        fighting the trends or fads unless they make specifically bad
        fits for specific situations.


        It's sort of like the library you have that I was looking at a
        little last night that bootstraps blueprint files without OSGi.
        That is simply another way of doing something that Spring,
        Guice, etc do it but it doesn't make OSGi or Spring obsolete.
        But it is nice to have in the toolbox.  I've been using
        blueprint for the past 4 years or so and being able to use a
        modified form of it for wiring up and booting little
        applications is nice.  I don't have to switch to a different
        dialect like Spring (although I used to use that heavily in the
        olden days).  So the MicroWebservices is just another way to use
        the standard CXF, Blueprint and OSGi mechanism to ease the pain
        of web service set up and configuration.

        As I get more, uhm, mature it is fun to watch how new fads come
        into being.  Docker and Kubernettes are the rage and I certainly
        understand the underlying technologies.  But how often do you
        see folks just grabbing hold of them because they are the
        latest, coolest thing?  I like Docker conceptually but working
        in karaf/blueprint/OSGi I'm not sure I have that many instances
        in my enterprise applications where I'd have a great need for
        them.  On the other hand, if I have a client insistent on them
        then seeing a tool like your blueprint without OSGi becomes far
        more interesting.


        On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider
        <[hidden email] <mailto:[hidden email]>> wrote:

            Thanks for the explanations. I think I now understand better.

            In effect it does a similar thing but with a different
            focus. Aries RSA and the CXF provider focus on the
            whiteboard pattern to kind of transparently export/import a
            service. Both do not have strong out of the box support for
            managing and centralizing configurations. It can be done but
            it less prepared than in your framework.
            This is an example of how to export a rest service:
            https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

            Your framework is more similar to the CXF blueprint
            namespace as you have to explicitly export a REST or SOAP
            service using blueprint. It focuses on the configuration
            aspect by managing a system wide and per service
            configuration. Something l really like there is the do a lot
            of the config system wide this is something I would like to
            add for Aries RSA too.

            I really like the whiteboard approach of RSA as it allows to
            have no or minimal dependencies from the user code to the
            framework. So maybe you are interested in combining the
            capabilities of both.
            RSA now has a custom service provider interface
            ExportPolicy. This is a hook where you can do system wide
            configuration and also things like auto exporting services
            based on annotations.

            I think you can achieve the same level of manageability in
            RSA as with your framework but it is not there already. If
            you are interested in helping out there I can help you to
            get into the code.

            Christian


            On 17.04.2016 01:08, Brad Johnson wrote:
            I'll have to look at the Aries RSA as I'm not as aware of
            it.  It doesn't appear to be similar to DOSGi from my
            experience with it.  I'll take a look at them again
            tomorrow. Like so many things in our OSGi, ServiceMix,
            Camel world there are about 10 different ways to do
            everything.  Some are better than others and most are
            better in certain circumstances.  This design is in
            reaction to consulting on a job that involves a company
            which is integrating multiple smaller companies that it
            owns and they all have a little different security,
            different ports, different web service requirements but
            are commonly going to be talking to the same Fuse instance
            in many (but not all) cases.


             This uses the OSGi mechanics for registration only and it
            doesn't require that the the interface itself be an OSGi
            service.  Here is a listener on a blueprint file for the
            MicroWebserviceManager.  I'm not filtering on any service
            property but but I certainly could.  The manager has a
            configuration associated with it for standard
            interceptors, etc.


            <reference-list id="microserviceListener"
            interface="org.enjekt.osgi.microserver.api.MicroWebservice">
            <reference-listener bind-method="register"
            unbind-method="unregister">
            <ref component-id="microserviceManager" />
            </reference-listener>
            </reference-list>


            The simple Echo bundle I have in there has this in its
            blueprint.

            <service
            interface="org.enjekt.osgi.microserver.api.MicroWebservice">
            <bean
            class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
            <argument value="org.enjekt.osgi.api.echo.EchoService" />
            <argument ref="impl" />
            <property name="restRelativeURI" value="/resources/echo" />
            <property name="soapRelativeURI" value="/services/echo" />

            </bean>
            </service>

            That MicroWebServiceRegistration inherits from the same
            class that the .MicroWebserviceManager uses for
            configuration.  When this simple Echo bundle starts it
            registers as a MicroWebService and the manager picks it up
            and combines the configuration found in both.  As you can
            see registration isn't specifying the host/port
            information so the one used by the MicroWebserviceManager
            will be used.  However, if I did specify a different
            host/port in the bundle's properties that would be used
            instead.  The manager itself never actually creates the
            server/endpoints.  It instantiates a container that it
            keeps in a map and that container instantiates the
            server/endpoints based on the combined information.

            Obviously if I uninstall the Echo bundle this particular
            MicroWebservice is unregistered, the manager is notified
            and it tells the container to shut down the endpoint.
            Those hard coded properties for restRelativeURI and
            soapRelativeURI would normally be part of the blueprint
            properties and so settable at runtime triggering a
            reload.  That in turn triggers the manager to tear down
            the old endpoint and start up a new one. Since the
            MicroWebserviceRegistration can accept the same sort of
            interceptors and providers that the manager uses I've used
            that as a way to register error/exceptionhandlers as well
            and in one case to override the JSON provider
            configuration.  One department wanted badgerfish and
            another wanted the root stripped and unwrapped.  Just
            specifying those differently in the sending bundle I could
            override the defaults.  If I wanted to I could also use
            different registrations to expose the same service on
            different relativeURIs or ports with different
            configuration information.  Here's an example of a service
            adding in an exception handler.

                            <property name="addProvider">
                                        <bean
            class="com.foo.internal.exceptionhandlers.MyExceptionHandler"
            />
                              </property

            If that's added to the manager's configuration then all
            endpoints that register with it get that exception
            handler. If it is registered with the bundle's
            MicroWebserviceRegistration then that only applies to it.
            In this case it is in an internal package but gets
            provided via the web service.  The
            MicroWebserviceRegistration also provides a UUID so when
            it the bundle is uninstalled and the manager gets notified
            of the fact it knows which container to pull from its
            registry and shut down.

            Just looking at that EchoService if I have three different
            divisions with different security and REST/SOAP
            requirements I can have three managers running via a
            simple blueprint set up of their service managers with
            service property filters for company A, company B, company
            C and when I register the EchoService I can specify
            different relative URIs but let each of the managers
            determine how the interface is actually going to be exposed.

            It works very well.  In practice I have most of the
            properties relevant to the bundle's requirements exposed
            in the cfg files making changes simple.

            I'll take a look at RSA tomorrow and see how it works.

            Brad

            On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider
            <<mailto:[hidden email]>[hidden email]
            <mailto:[hidden email]>> wrote:

                Hi Brad,

                from what I understood it allows to expose OSGi
                services as SOAP and REST endpoints. This sounds very
                similar to Aries RSA and CXF DOSGi. Can you explain
                what it does differently?

                Christian

                2016-04-16 17:39 GMT+02:00 Brad Johnson
                <<mailto:[hidden email]>[hidden email]
                <mailto:[hidden email]>>:

                    https://github.com/Ranx0r0x/Enjekt-Microservice

                    I've started a project that may or may not be of
                    interest to the community.  I've been finding it
                    of great benefit in my current projects where I
                    combine it with Camel for certain items.  One
                    change I plan on making as soon as I can get to it
                    is to change the BOM from importing Fuse
                    dependencies to Servicemix.  That should be
                    relatively simple but I'm working with it in Fuse
                    currently so it hasn't been a high priority.

                    Any feedback is appreciated.

                    Brad




                --
                --
                Christian Schneider
                http://www.liquid-reality.de
                <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>

                Open Source Architect
                http://www.talend.com
                <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>




            --
            Christian Schneider
            http://www.liquid-reality.de

            Open Source Architect
            http://www.talend.com





    --
    --
    Christian Schneider
    http://www.liquid-reality.de
    <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>

    Open Source Architect
    http://www.talend.com
    <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>



--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com

Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
Christian,

I just re-read your section on the static profiles.  That makes sense.  I could see this becoming like puppet/chef sort of recipes or even like the way Docker allows building up of images.  Not that I know those that well.  I use features all the time of course but see huge advantage to making that a bundle time set of static steps instead of the current runtime usage.  Not that the current use is bad, mind you, its context is different and slimming down the karaf core by eliminating that management overhead would be good for something like karaf-boot.  In the current monolithic enterprise environment it makes sense to have stacks of features available to load when necessary.  The static mechanism would get rid of that.  But it would also permit building up a centralized registry or library of features that one could leverage by adding into a boot recipe of some sort.   Maybe we'd get the quick flexibility for creating projects that archetypes always seemed to promise but never quit seemed to manage.

One item that has always bothered me about features is they are orthogonal to but replicate much of what goes into a POM.   One ends up with two sets of dependency management mechanisms that have never really dovetailed.  It would be nice if we had a Maven plugin that could look at the dependencies in a POM and create a static feature profile or at least give a good guess at what they should be while allowing for some tweaking.  Perhaps since the karaf-boot environment is static and doesn't rely on or expect another environment to provide dependencies that would be easier to accomplish.

Funny how Moore's law took a sideways turn on us.  Now we don't have the ever increasing clock speed but we have cores coming out our ears and RAM and disk space in abundance.  A little fatness in our deployments is an acceptable trade off now.

Brad 



On Sun, Apr 17, 2016 at 5:25 PM, Brad Johnson <[hidden email]> wrote:
Sounds like you both have your reservations about using the noOSGi version of blueprint and you obviously know it better than the rest of us.  When do you anticipate something like Karaf boot working?  I think the Karaf site has 7/15 as the target date for Cellar and 6/16 for karaf 4.1. 

https://karaf.apache.org/projects.html#boot

Since boot relies on Cellar I'm guess that some of the work going into those dates are in support of this project.  It looks like a happy day.

The current size of Karaf is about 20MB so it certainly should be slim enough for the sort of enterprise containers that might be used via Karaf boot.  It'll be interesting to see where some of the lines get drawn.  I've read some on Docker and played with it a little but don't know it well enough.  But I could see cases where each image didn't contain its own Karaf but used a container at a more global/OS level in the way Docker shares other parts of the OS or where it might be specified as a Karaf instance wrapped in each image itself.  Guess I'm going to have to get more knowledgeable about it.

I'll display some frank ignorance here.  When I move up out of the OSGi container world I've lived in for the past number of years with Camel and blueprint some of the areas where concerns are competitive and where they are simply orthogonal to one another is blurry.  Kubernetes is an obvious player in this market and I'm sure that if work is on-going with Karaf-boot there must be plans in place for it as well.

As always, our software world is an exciting and confusing world.

Brad

On Sun, Apr 17, 2016 at 2:18 PM, Jean-Baptiste Onofré <[hidden email]> wrote:
Hi Brad,

I second Christian. I think that leveraging OSGi services is very interesting to build non-monolithic microservices (largely better than big monolithic approach).

As Christian said, karaf-boot should help a lot there as it will:

1. Simplify the way of building application thanks to annotation
2. Create distribution as single jar, profile, dockerfile, docker image, etc. It's the "run everywhere" paradign. Right now, you have first to install Karaf and then deploy your application in Karaf. With karaf-boot, you focus on your business code, and karaf-boot deals with the rest (up to the distribution).

Regards
JB


On 04/17/2016 06:15 PM, Brad Johnson wrote:
I think you hit it with your observation about why even use that
MicroWebServer component.  Different requirements on different addresses
and customizations.  Also the bundle itself can come and go and have its
web services deploy and undeploy without having to define server
endpoints in the bundle itself.  If my EchoService bundle is installed
and there isn't MicroWebServer installed it simply installs as another
OSGi bundle whose local service interface might be used by other
bundles.  But it isn't itself defining any CXF server information that
may or may not be wanted at deployment time.  It separates the two
concerns a bit.

I'm not in the rush or on the bandwagon to adopt Docker but one does
have to be prepared.  The one thing I'd like about having a NoOSGi
version of blueprint is not having to jump between different XML
bootstrap mechanics - Spring or Guice then back to Blueprint.  Certainly
Spring is close enough to Blueprint that the syntax is easy enough but
now one has different mechanics all the way from XML up through builds.
Not a huge deal but if I can make small applications with Blueprint that
just eases life a little.  I'm in an environment right now where Fuse is
the central focal point so the OSGi, Camel, Blueprint are not going away
anytime soon. But I'm also getting where other departments want us to
develop and deploy small self-contained services for them.  As you point
out, even a minimal karaf/servicemix implementation can be big for
that.  I also have the irritant of dealing with licensing issues because
of the corporate environment.  I can't just plop minimal installs of
Fuse in different divisions because of licensing.  I say it's irritating
only because that means I have to fish for technical solutions that
aren't addressing a technical or business issue but a legal one.

But initially it will be for testing as being able to set up stub
services for code to call out to will be quite handy.

On Sun, Apr 17, 2016 at 10:42 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I think the main feature of the MicroWebServer approach you show is
    that you can define multiple endpoints for the same impl. This is
    indeed a niche that Aries RSA does not fill and does not try to
    fill. So it might be a good case for people to use it.

    About blueprint without OSGi.. I think it is great for testing but
    if your application really is not OSGi based I would not bother with
    it and just use spring or another non OSGi dependency injection.

    Microservices and docker are indeed very popular at the moment.
    Still I think it makes sense to use OSGi. On the other hand it would
    also be overkill to use a full blown karaf server for it as you will
    only host one application per process.

    We are currently working on karaf boot which attempts to support
    this style very well. It will support a mainly annotation based
    developement model like spring boot. On the packaging side you can
    use the static karaf profile to package your application using karaf
    features into a fixed set of bundles at build time. So at runtime
    you still have karaf but without the feature service so the runtime
    is simpler and nicely suited to host one single application.

    Christian

    2016-04-17 17:14 GMT+02:00 Brad Johnson
    <[hidden email] <mailto:[hidden email]>>:


        I took a look at RSA and DOSGi again.  I'd forgotten the RSA
        acronym so didn't recognize that I had looked at it before.  My
        conclusion is very similar to your own.  They are orthogonal
        concerns with a small amount of overlap..  I think what would
        make the determination of which one to use, for me, would be a
        very explicit context.  Obviously what I made isn't really about
        remote management or distributed architecture.  It is more
        tactical than that and related to specific bundles.  So if
        someone were to ask me about a solution for a server farm the
        MicroWebservice would NOT be on the candidate list. The
        MicroWebservice does use the OSGi whiteboard to listen for the
        MicroWebservice registration events. Because it always uses the
        same OSGi registration service interface 1 .. N MicroWebservers
        can listen for the same events with their list-listeners and you
        could filter on service properties.  I forget the LDAP filtering
        syntax off the top of my head.

        I just got tired of having to set up and re-set up so many
        different blueprint configurations fro different endpoints.  And
        if you try to centralize that in something like gateway bundle
        now you have an issue that when the provider bundle is
        undeployed the web service is still hanging out there in space
        disconnected.  With this mechanic if you undeploy your bundle
        all the standard Aries blueprint mechanics trigger the removal
        of the web services associated with a given bundle.  But all of
        that relies on your Aries OSGi mechanisms with blueprint to
        function.  It can't exist without it.  That's part of why I
        thought of mentioning it here first as it might actually fill a
        niche need for Aries or Aries users.  If folks here got
        interested in it then maybe you'd adopt it some day.  Who
        knows.  But at least I'd get more eyes and thoughts and
        solidification to it as well.  And if folks don't find it fits
        any need for them, then I won't waste a lot of time on it either.

        For better or worse microservices are the fashion today.  As a
        consultant I don't get to make the call and don't even bother
        fighting the trends or fads unless they make specifically bad
        fits for specific situations.


        It's sort of like the library you have that I was looking at a
        little last night that bootstraps blueprint files without OSGi.
        That is simply another way of doing something that Spring,
        Guice, etc do it but it doesn't make OSGi or Spring obsolete.
        But it is nice to have in the toolbox.  I've been using
        blueprint for the past 4 years or so and being able to use a
        modified form of it for wiring up and booting little
        applications is nice.  I don't have to switch to a different
        dialect like Spring (although I used to use that heavily in the
        olden days).  So the MicroWebservices is just another way to use
        the standard CXF, Blueprint and OSGi mechanism to ease the pain
        of web service set up and configuration.

        As I get more, uhm, mature it is fun to watch how new fads come
        into being.  Docker and Kubernettes are the rage and I certainly
        understand the underlying technologies.  But how often do you
        see folks just grabbing hold of them because they are the
        latest, coolest thing?  I like Docker conceptually but working
        in karaf/blueprint/OSGi I'm not sure I have that many instances
        in my enterprise applications where I'd have a great need for
        them.  On the other hand, if I have a client insistent on them
        then seeing a tool like your blueprint without OSGi becomes far
        more interesting.


        On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider
        <[hidden email] <mailto:[hidden email]>> wrote:

            Thanks for the explanations. I think I now understand better.

            In effect it does a similar thing but with a different
            focus. Aries RSA and the CXF provider focus on the
            whiteboard pattern to kind of transparently export/import a
            service. Both do not have strong out of the box support for
            managing and centralizing configurations. It can be done but
            it less prepared than in your framework.
            This is an example of how to export a rest service:
            https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java

            Your framework is more similar to the CXF blueprint
            namespace as you have to explicitly export a REST or SOAP
            service using blueprint. It focuses on the configuration
            aspect by managing a system wide and per service
            configuration. Something l really like there is the do a lot
            of the config system wide this is something I would like to
            add for Aries RSA too.

            I really like the whiteboard approach of RSA as it allows to
            have no or minimal dependencies from the user code to the
            framework. So maybe you are interested in combining the
            capabilities of both.
            RSA now has a custom service provider interface
            ExportPolicy. This is a hook where you can do system wide
            configuration and also things like auto exporting services
            based on annotations.

            I think you can achieve the same level of manageability in
            RSA as with your framework but it is not there already. If
            you are interested in helping out there I can help you to
            get into the code.

            Christian


            On 17.04.2016 01:08, Brad Johnson wrote:
            I'll have to look at the Aries RSA as I'm not as aware of
            it.  It doesn't appear to be similar to DOSGi from my
            experience with it.  I'll take a look at them again
            tomorrow. Like so many things in our OSGi, ServiceMix,
            Camel world there are about 10 different ways to do
            everything.  Some are better than others and most are
            better in certain circumstances.  This design is in
            reaction to consulting on a job that involves a company
            which is integrating multiple smaller companies that it
            owns and they all have a little different security,
            different ports, different web service requirements but
            are commonly going to be talking to the same Fuse instance
            in many (but not all) cases.


             This uses the OSGi mechanics for registration only and it
            doesn't require that the the interface itself be an OSGi
            service.  Here is a listener on a blueprint file for the
            MicroWebserviceManager.  I'm not filtering on any service
            property but but I certainly could.  The manager has a
            configuration associated with it for standard
            interceptors, etc.


            <reference-list id="microserviceListener"
            interface="org.enjekt.osgi.microserver.api.MicroWebservice">
            <reference-listener bind-method="register"
            unbind-method="unregister">
            <ref component-id="microserviceManager" />
            </reference-listener>
            </reference-list>


            The simple Echo bundle I have in there has this in its
            blueprint.

            <service
            interface="org.enjekt.osgi.microserver.api.MicroWebservice">
            <bean
            class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
            <argument value="org.enjekt.osgi.api.echo.EchoService" />
            <argument ref="impl" />
            <property name="restRelativeURI" value="/resources/echo" />
            <property name="soapRelativeURI" value="/services/echo" />

            </bean>
            </service>

            That MicroWebServiceRegistration inherits from the same
            class that the .MicroWebserviceManager uses for
            configuration.  When this simple Echo bundle starts it
            registers as a MicroWebService and the manager picks it up
            and combines the configuration found in both.  As you can
            see registration isn't specifying the host/port
            information so the one used by the MicroWebserviceManager
            will be used.  However, if I did specify a different
            host/port in the bundle's properties that would be used
            instead.  The manager itself never actually creates the
            server/endpoints.  It instantiates a container that it
            keeps in a map and that container instantiates the
            server/endpoints based on the combined information.

            Obviously if I uninstall the Echo bundle this particular
            MicroWebservice is unregistered, the manager is notified
            and it tells the container to shut down the endpoint.
            Those hard coded properties for restRelativeURI and
            soapRelativeURI would normally be part of the blueprint
            properties and so settable at runtime triggering a
            reload.  That in turn triggers the manager to tear down
            the old endpoint and start up a new one. Since the
            MicroWebserviceRegistration can accept the same sort of
            interceptors and providers that the manager uses I've used
            that as a way to register error/exceptionhandlers as well
            and in one case to override the JSON provider
            configuration.  One department wanted badgerfish and
            another wanted the root stripped and unwrapped.  Just
            specifying those differently in the sending bundle I could
            override the defaults.  If I wanted to I could also use
            different registrations to expose the same service on
            different relativeURIs or ports with different
            configuration information.  Here's an example of a service
            adding in an exception handler.

                            <property name="addProvider">
                                        <bean
            class="com.foo.internal.exceptionhandlers.MyExceptionHandler"
            />
                              </property

            If that's added to the manager's configuration then all
            endpoints that register with it get that exception
            handler. If it is registered with the bundle's
            MicroWebserviceRegistration then that only applies to it.
            In this case it is in an internal package but gets
            provided via the web service.  The
            MicroWebserviceRegistration also provides a UUID so when
            it the bundle is uninstalled and the manager gets notified
            of the fact it knows which container to pull from its
            registry and shut down.

            Just looking at that EchoService if I have three different
            divisions with different security and REST/SOAP
            requirements I can have three managers running via a
            simple blueprint set up of their service managers with
            service property filters for company A, company B, company
            C and when I register the EchoService I can specify
            different relative URIs but let each of the managers
            determine how the interface is actually going to be exposed.

            It works very well.  In practice I have most of the
            properties relevant to the bundle's requirements exposed
            in the cfg files making changes simple.

            I'll take a look at RSA tomorrow and see how it works.

            Brad

            On Sat, Apr 16, 2016 at 1:42 PM, Christian Schneider
            <<mailto:[hidden email]>[hidden email]
            <mailto:[hidden email]>> wrote:

                Hi Brad,

                from what I understood it allows to expose OSGi
                services as SOAP and REST endpoints. This sounds very
                similar to Aries RSA and CXF DOSGi. Can you explain
                what it does differently?

                Christian

                2016-04-16 17:39 GMT+02:00 Brad Johnson
                <<mailto:[hidden email]>[hidden email]
                <mailto:[hidden email]>>:

                    https://github.com/Ranx0r0x/Enjekt-Microservice

                    I've started a project that may or may not be of
                    interest to the community.  I've been finding it
                    of great benefit in my current projects where I
                    combine it with Camel for certain items.  One
                    change I plan on making as soon as I can get to it
                    is to change the BOM from importing Fuse
                    dependencies to Servicemix.  That should be
                    relatively simple but I'm working with it in Fuse
                    currently so it hasn't been a high priority.

                    Any feedback is appreciated.

                    Brad




                --
                --
                Christian Schneider
                http://www.liquid-reality.de
                <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>

                Open Source Architect
                http://www.talend.com
                <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>




            --
            Christian Schneider
            http://www.liquid-reality.de

            Open Source Architect
            http://www.talend.com





    --
    --
    Christian Schneider
    http://www.liquid-reality.de
    <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>

    Open Source Architect
    http://www.talend.com
    <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>



--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com


Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Jean-Baptiste Onofré
Hi Brad,

About the features tight to POM, I don't think it's a big deal as:

1. you can use maven property in features (it's what we do internally in
Karaf)
2. the karaf-maven-plugin can help you there (with the generate goal)

Regards
JB

On 04/18/2016 07:24 AM, Brad Johnson wrote:

> Christian,
>
> I just re-read your section on the static profiles.  That makes sense.
> I could see this becoming like puppet/chef sort of recipes or even like
> the way Docker allows building up of images.  Not that I know those that
> well.  I use features all the time of course but see huge advantage to
> making that a bundle time set of static steps instead of the current
> runtime usage.  Not that the current use is bad, mind you, its context
> is different and slimming down the karaf core by eliminating that
> management overhead would be good for something like karaf-boot.  In the
> current monolithic enterprise environment it makes sense to have stacks
> of features available to load when necessary.  The static mechanism
> would get rid of that.  But it would also permit building up a
> centralized registry or library of features that one could leverage by
> adding into a boot recipe of some sort.   Maybe we'd get the quick
> flexibility for creating projects that archetypes always seemed to
> promise but never quit seemed to manage.
>
> One item that has always bothered me about features is they are
> orthogonal to but replicate much of what goes into a POM.   One ends up
> with two sets of dependency management mechanisms that have never really
> dovetailed.  It would be nice if we had a Maven plugin that could look
> at the dependencies in a POM and create a static feature profile or at
> least give a good guess at what they should be while allowing for some
> tweaking.  Perhaps since the karaf-boot environment is static and
> doesn't rely on or expect another environment to provide dependencies
> that would be easier to accomplish.
>
> Funny how Moore's law took a sideways turn on us.  Now we don't have the
> ever increasing clock speed but we have cores coming out our ears and
> RAM and disk space in abundance.  A little fatness in our deployments is
> an acceptable trade off now.
>
> Brad
>
>
>
> On Sun, Apr 17, 2016 at 5:25 PM, Brad Johnson
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Sounds like you both have your reservations about using the noOSGi
>     version of blueprint and you obviously know it better than the rest
>     of us.  When do you anticipate something like Karaf boot working?  I
>     think the Karaf site has 7/15 as the target date for Cellar and 6/16
>     for karaf 4.1.
>
>     https://karaf.apache.org/projects.html#boot
>
>     Since boot relies on Cellar I'm guess that some of the work going
>     into those dates are in support of this project.  It looks like a
>     happy day.
>
>     The current size of Karaf is about 20MB so it certainly should be
>     slim enough for the sort of enterprise containers that might be used
>     via Karaf boot.  It'll be interesting to see where some of the lines
>     get drawn.  I've read some on Docker and played with it a little but
>     don't know it well enough.  But I could see cases where each image
>     didn't contain its own Karaf but used a container at a more
>     global/OS level in the way Docker shares other parts of the OS or
>     where it might be specified as a Karaf instance wrapped in each
>     image itself.  Guess I'm going to have to get more knowledgeable
>     about it.
>
>     I'll display some frank ignorance here.  When I move up out of the
>     OSGi container world I've lived in for the past number of years with
>     Camel and blueprint some of the areas where concerns are competitive
>     and where they are simply orthogonal to one another is blurry.
>     Kubernetes is an obvious player in this market and I'm sure that if
>     work is on-going with Karaf-boot there must be plans in place for it
>     as well.
>
>     As always, our software world is an exciting and confusing world.
>
>     Brad
>
>     On Sun, Apr 17, 2016 at 2:18 PM, Jean-Baptiste Onofré
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>         Hi Brad,
>
>         I second Christian. I think that leveraging OSGi services is
>         very interesting to build non-monolithic microservices (largely
>         better than big monolithic approach).
>
>         As Christian said, karaf-boot should help a lot there as it will:
>
>         1. Simplify the way of building application thanks to annotation
>         2. Create distribution as single jar, profile, dockerfile,
>         docker image, etc. It's the "run everywhere" paradign. Right
>         now, you have first to install Karaf and then deploy your
>         application in Karaf. With karaf-boot, you focus on your
>         business code, and karaf-boot deals with the rest (up to the
>         distribution).
>
>         Regards
>         JB
>
>
>         On 04/17/2016 06:15 PM, Brad Johnson wrote:
>
>             I think you hit it with your observation about why even use that
>             MicroWebServer component.  Different requirements on
>             different addresses
>             and customizations.  Also the bundle itself can come and go
>             and have its
>             web services deploy and undeploy without having to define server
>             endpoints in the bundle itself.  If my EchoService bundle is
>             installed
>             and there isn't MicroWebServer installed it simply installs
>             as another
>             OSGi bundle whose local service interface might be used by other
>             bundles.  But it isn't itself defining any CXF server
>             information that
>             may or may not be wanted at deployment time.  It separates
>             the two
>             concerns a bit.
>
>             I'm not in the rush or on the bandwagon to adopt Docker but
>             one does
>             have to be prepared.  The one thing I'd like about having a
>             NoOSGi
>             version of blueprint is not having to jump between different XML
>             bootstrap mechanics - Spring or Guice then back to
>             Blueprint.  Certainly
>             Spring is close enough to Blueprint that the syntax is easy
>             enough but
>             now one has different mechanics all the way from XML up
>             through builds.
>             Not a huge deal but if I can make small applications with
>             Blueprint that
>             just eases life a little.  I'm in an environment right now
>             where Fuse is
>             the central focal point so the OSGi, Camel, Blueprint are
>             not going away
>             anytime soon. But I'm also getting where other departments
>             want us to
>             develop and deploy small self-contained services for them.
>             As you point
>             out, even a minimal karaf/servicemix implementation can be
>             big for
>             that.  I also have the irritant of dealing with licensing
>             issues because
>             of the corporate environment.  I can't just plop minimal
>             installs of
>             Fuse in different divisions because of licensing.  I say
>             it's irritating
>             only because that means I have to fish for technical
>             solutions that
>             aren't addressing a technical or business issue but a legal one.
>
>             But initially it will be for testing as being able to set up
>             stub
>             services for code to call out to will be quite handy.
>
>             On Sun, Apr 17, 2016 at 10:42 AM, Christian Schneider
>             <[hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email]
>             <mailto:[hidden email]>>> wrote:
>
>                  I think the main feature of the MicroWebServer approach
>             you show is
>                  that you can define multiple endpoints for the same
>             impl. This is
>                  indeed a niche that Aries RSA does not fill and does
>             not try to
>                  fill. So it might be a good case for people to use it.
>
>                  About blueprint without OSGi.. I think it is great for
>             testing but
>                  if your application really is not OSGi based I would
>             not bother with
>                  it and just use spring or another non OSGi dependency
>             injection.
>
>                  Microservices and docker are indeed very popular at the
>             moment.
>                  Still I think it makes sense to use OSGi. On the other
>             hand it would
>                  also be overkill to use a full blown karaf server for
>             it as you will
>                  only host one application per process.
>
>                  We are currently working on karaf boot which attempts
>             to support
>                  this style very well. It will support a mainly
>             annotation based
>                  developement model like spring boot. On the packaging
>             side you can
>                  use the static karaf profile to package your
>             application using karaf
>                  features into a fixed set of bundles at build time. So
>             at runtime
>                  you still have karaf but without the feature service so
>             the runtime
>                  is simpler and nicely suited to host one single
>             application.
>
>                  Christian
>
>                  2016-04-17 17:14 GMT+02:00 Brad Johnson
>                  <[hidden email]
>             <mailto:[hidden email]>
>             <mailto:[hidden email]
>             <mailto:[hidden email]>>>:
>
>
>                      I took a look at RSA and DOSGi again.  I'd
>             forgotten the RSA
>                      acronym so didn't recognize that I had looked at it
>             before.  My
>                      conclusion is very similar to your own.  They are
>             orthogonal
>                      concerns with a small amount of overlap..  I think
>             what would
>                      make the determination of which one to use, for me,
>             would be a
>                      very explicit context.  Obviously what I made isn't
>             really about
>                      remote management or distributed architecture.  It
>             is more
>                      tactical than that and related to specific
>             bundles.  So if
>                      someone were to ask me about a solution for a
>             server farm the
>                      MicroWebservice would NOT be on the candidate list. The
>                      MicroWebservice does use the OSGi whiteboard to
>             listen for the
>                      MicroWebservice registration events. Because it
>             always uses the
>                      same OSGi registration service interface 1 .. N
>             MicroWebservers
>                      can listen for the same events with their
>             list-listeners and you
>                      could filter on service properties.  I forget the
>             LDAP filtering
>                      syntax off the top of my head.
>
>                      I just got tired of having to set up and re-set up
>             so many
>                      different blueprint configurations fro different
>             endpoints.  And
>                      if you try to centralize that in something like
>             gateway bundle
>                      now you have an issue that when the provider bundle is
>                      undeployed the web service is still hanging out
>             there in space
>                      disconnected.  With this mechanic if you undeploy
>             your bundle
>                      all the standard Aries blueprint mechanics trigger
>             the removal
>                      of the web services associated with a given
>             bundle.  But all of
>                      that relies on your Aries OSGi mechanisms with
>             blueprint to
>                      function.  It can't exist without it.  That's part
>             of why I
>                      thought of mentioning it here first as it might
>             actually fill a
>                      niche need for Aries or Aries users.  If folks here got
>                      interested in it then maybe you'd adopt it some
>             day.  Who
>                      knows.  But at least I'd get more eyes and thoughts and
>                      solidification to it as well.  And if folks don't
>             find it fits
>                      any need for them, then I won't waste a lot of time
>             on it either.
>
>                      For better or worse microservices are the fashion
>             today.  As a
>                      consultant I don't get to make the call and don't
>             even bother
>                      fighting the trends or fads unless they make
>             specifically bad
>                      fits for specific situations.
>
>
>                      It's sort of like the library you have that I was
>             looking at a
>                      little last night that bootstraps blueprint files
>             without OSGi.
>                      That is simply another way of doing something that
>             Spring,
>                      Guice, etc do it but it doesn't make OSGi or Spring
>             obsolete.
>                      But it is nice to have in the toolbox.  I've been using
>                      blueprint for the past 4 years or so and being able
>             to use a
>                      modified form of it for wiring up and booting little
>                      applications is nice.  I don't have to switch to a
>             different
>                      dialect like Spring (although I used to use that
>             heavily in the
>                      olden days).  So the MicroWebservices is just
>             another way to use
>                      the standard CXF, Blueprint and OSGi mechanism to
>             ease the pain
>                      of web service set up and configuration.
>
>                      As I get more, uhm, mature it is fun to watch how
>             new fads come
>                      into being.  Docker and Kubernettes are the rage
>             and I certainly
>                      understand the underlying technologies.  But how
>             often do you
>                      see folks just grabbing hold of them because they
>             are the
>                      latest, coolest thing?  I like Docker conceptually
>             but working
>                      in karaf/blueprint/OSGi I'm not sure I have that
>             many instances
>                      in my enterprise applications where I'd have a
>             great need for
>                      them.  On the other hand, if I have a client
>             insistent on them
>                      then seeing a tool like your blueprint without OSGi
>             becomes far
>                      more interesting.
>
>
>                      On Sun, Apr 17, 2016 at 7:01 AM, Christian Schneider
>                      <[hidden email]
>             <mailto:[hidden email]>
>             <mailto:[hidden email]
>             <mailto:[hidden email]>>> wrote:
>
>                          Thanks for the explanations. I think I now
>             understand better.
>
>                          In effect it does a similar thing but with a
>             different
>                          focus. Aries RSA and the CXF provider focus on the
>                          whiteboard pattern to kind of transparently
>             export/import a
>                          service. Both do not have strong out of the box
>             support for
>                          managing and centralizing configurations. It
>             can be done but
>                          it less prepared than in your framework.
>                          This is an example of how to export a rest service:
>             https://github.com/cschneider/Karaf-Tutorial/blob/master/tasklist-ds/service/src/main/java/net/lr/tasklist/service/TaskServiceRest.java
>
>                          Your framework is more similar to the CXF blueprint
>                          namespace as you have to explicitly export a
>             REST or SOAP
>                          service using blueprint. It focuses on the
>             configuration
>                          aspect by managing a system wide and per service
>                          configuration. Something l really like there is
>             the do a lot
>                          of the config system wide this is something I
>             would like to
>                          add for Aries RSA too.
>
>                          I really like the whiteboard approach of RSA as
>             it allows to
>                          have no or minimal dependencies from the user
>             code to the
>                          framework. So maybe you are interested in
>             combining the
>                          capabilities of both.
>                          RSA now has a custom service provider interface
>                          ExportPolicy. This is a hook where you can do
>             system wide
>                          configuration and also things like auto
>             exporting services
>                          based on annotations.
>
>                          I think you can achieve the same level of
>             manageability in
>                          RSA as with your framework but it is not there
>             already. If
>                          you are interested in helping out there I can
>             help you to
>                          get into the code.
>
>                          Christian
>
>
>                          On 17.04.2016 01:08, Brad Johnson wrote:
>
>                              I'll have to look at the Aries RSA as I'm
>                 not as aware of
>                              it.  It doesn't appear to be similar to
>                 DOSGi from my
>                              experience with it.  I'll take a look at
>                 them again
>                              tomorrow. Like so many things in our OSGi,
>                 ServiceMix,
>                              Camel world there are about 10 different
>                 ways to do
>                              everything.  Some are better than others
>                 and most are
>                              better in certain circumstances.  This
>                 design is in
>                              reaction to consulting on a job that
>                 involves a company
>                              which is integrating multiple smaller
>                 companies that it
>                              owns and they all have a little different
>                 security,
>                              different ports, different web service
>                 requirements but
>                              are commonly going to be talking to the
>                 same Fuse instance
>                              in many (but not all) cases.
>
>
>                               This uses the OSGi mechanics for
>                 registration only and it
>                              doesn't require that the the interface
>                 itself be an OSGi
>                              service.  Here is a listener on a blueprint
>                 file for the
>                              MicroWebserviceManager.  I'm not filtering
>                 on any service
>                              property but but I certainly could.  The
>                 manager has a
>                              configuration associated with it for standard
>                              interceptors, etc.
>
>
>                              <reference-list id="microserviceListener"
>
>                 interface="org.enjekt.osgi.microserver.api.MicroWebservice">
>                              <reference-listener bind-method="register"
>                              unbind-method="unregister">
>                              <ref component-id="microserviceManager" />
>                              </reference-listener>
>                              </reference-list>
>
>
>                              The simple Echo bundle I have in there has
>                 this in its
>                              blueprint.
>
>                              <service
>
>                 interface="org.enjekt.osgi.microserver.api.MicroWebservice">
>                              <bean
>
>                 class="org.enjekt.osgi.microserver.impl.MicroWebserviceRegistration">
>                              <argument
>                 value="org.enjekt.osgi.api.echo.EchoService" />
>                              <argument ref="impl" />
>                              <property name="restRelativeURI"
>                 value="/resources/echo" />
>                              <property name="soapRelativeURI"
>                 value="/services/echo" />
>
>                              </bean>
>                              </service>
>
>                              That MicroWebServiceRegistration inherits
>                 from the same
>                              class that the .MicroWebserviceManager uses for
>                              configuration.  When this simple Echo
>                 bundle starts it
>                              registers as a MicroWebService and the
>                 manager picks it up
>                              and combines the configuration found in
>                 both.  As you can
>                              see registration isn't specifying the host/port
>                              information so the one used by the
>                 MicroWebserviceManager
>                              will be used.  However, if I did specify a
>                 different
>                              host/port in the bundle's properties that
>                 would be used
>                              instead.  The manager itself never actually
>                 creates the
>                              server/endpoints.  It instantiates a
>                 container that it
>                              keeps in a map and that container
>                 instantiates the
>                              server/endpoints based on the combined
>                 information.
>
>                              Obviously if I uninstall the Echo bundle
>                 this particular
>                              MicroWebservice is unregistered, the
>                 manager is notified
>                              and it tells the container to shut down the
>                 endpoint.
>                              Those hard coded properties for
>                 restRelativeURI and
>                              soapRelativeURI would normally be part of
>                 the blueprint
>                              properties and so settable at runtime
>                 triggering a
>                              reload.  That in turn triggers the manager
>                 to tear down
>                              the old endpoint and start up a new one.
>                 Since the
>                              MicroWebserviceRegistration can accept the
>                 same sort of
>                              interceptors and providers that the manager
>                 uses I've used
>                              that as a way to register
>                 error/exceptionhandlers as well
>                              and in one case to override the JSON provider
>                              configuration.  One department wanted
>                 badgerfish and
>                              another wanted the root stripped and
>                 unwrapped.  Just
>                              specifying those differently in the sending
>                 bundle I could
>                              override the defaults.  If I wanted to I
>                 could also use
>                              different registrations to expose the same
>                 service on
>                              different relativeURIs or ports with different
>                              configuration information.  Here's an
>                 example of a service
>                              adding in an exception handler.
>
>                                              <property name="addProvider">
>                                                          <bean
>
>                 class="com.foo.internal.exceptionhandlers.MyExceptionHandler"
>                              />
>                                                </property
>
>                              If that's added to the manager's
>                 configuration then all
>                              endpoints that register with it get that
>                 exception
>                              handler. If it is registered with the bundle's
>                              MicroWebserviceRegistration then that only
>                 applies to it.
>                              In this case it is in an internal package
>                 but gets
>                              provided via the web service.  The
>                              MicroWebserviceRegistration also provides a
>                 UUID so when
>                              it the bundle is uninstalled and the
>                 manager gets notified
>                              of the fact it knows which container to
>                 pull from its
>                              registry and shut down.
>
>                              Just looking at that EchoService if I have
>                 three different
>                              divisions with different security and REST/SOAP
>                              requirements I can have three managers
>                 running via a
>                              simple blueprint set up of their service
>                 managers with
>                              service property filters for company A,
>                 company B, company
>                              C and when I register the EchoService I can
>                 specify
>                              different relative URIs but let each of the
>                 managers
>                              determine how the interface is actually
>                 going to be exposed.
>
>                              It works very well.  In practice I have
>                 most of the
>                              properties relevant to the bundle's
>                 requirements exposed
>                              in the cfg files making changes simple.
>
>                              I'll take a look at RSA tomorrow and see
>                 how it works.
>
>                              Brad
>
>                              On Sat, Apr 16, 2016 at 1:42 PM, Christian
>                 Schneider
>                              <<mailto:[hidden email]
>                 <mailto:[hidden email]>>[hidden email]
>                 <mailto:[hidden email]>
>                              <mailto:[hidden email]
>                 <mailto:[hidden email]>>> wrote:
>
>                                  Hi Brad,
>
>                                  from what I understood it allows to
>                 expose OSGi
>                                  services as SOAP and REST endpoints.
>                 This sounds very
>                                  similar to Aries RSA and CXF DOSGi. Can
>                 you explain
>                                  what it does differently?
>
>                                  Christian
>
>                                  2016-04-16 17:39 GMT+02:00 Brad Johnson
>                                  <<mailto:[hidden email]
>                 <mailto:[hidden email]>>[hidden email]
>                 <mailto:[hidden email]>
>                                  <mailto:[hidden email]
>                 <mailto:[hidden email]>>>:
>
>                 https://github.com/Ranx0r0x/Enjekt-Microservice
>
>                                      I've started a project that may or
>                 may not be of
>                                      interest to the community.  I've
>                 been finding it
>                                      of great benefit in my current
>                 projects where I
>                                      combine it with Camel for certain
>                 items.  One
>                                      change I plan on making as soon as
>                 I can get to it
>                                      is to change the BOM from importing
>                 Fuse
>                                      dependencies to Servicemix.  That
>                 should be
>                                      relatively simple but I'm working
>                 with it in Fuse
>                                      currently so it hasn't been a high
>                 priority.
>
>                                      Any feedback is appreciated.
>
>                                      Brad
>
>
>
>
>                                  --
>                                  --
>                                  Christian Schneider
>                 http://www.liquid-reality.de
>
>                 <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
>
>                                  Open Source Architect
>                 http://www.talend.com
>
>                 <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>
>
>
>
>
>                          --
>                          Christian Schneider
>             http://www.liquid-reality.de
>
>                          Open Source Architect
>             http://www.talend.com
>
>
>
>
>
>                  --
>                  --
>                  Christian Schneider
>             http://www.liquid-reality.de
>
>             <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>
>
>                  Open Source Architect
>             http://www.talend.com
>
>             <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.talend.com>
>
>
>
>         --
>         Jean-Baptiste Onofré
>         [hidden email] <mailto:[hidden email]>
>         http://blog.nanthrax.net
>         Talend - http://www.talend.com
>
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Christian Schneider
In reply to this post by Brad Johnson
I am not very familiar with the concept of profiles till now. Guillaume
explained it a bit to me but honestly I hope we do not need it too
often. An OSGi application should be mainly built out of bundles and
profiles concentrate more on the non OSGi static files in karaf.

The main part of the packaging in karaf boot is to resolve features at
build time. It uses the static profile but nothing else from the
profiles concept. This part already works. I have created such a
packaging for the tasklist-ds example:
https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

The second part. The karaf boot development model is still in the making
but you can already use the packaging if you want.

About features and poms you are right. You have to maintain two sets of
dependencies but this is just a result of the fact that OSGi resolves
differently from maven. So this never will completely go away. What we
could do with a pom is create an obr repository. This could then be used
to back a feature repo. So currently in a feature we have to list all
the bundles, with a backing obr repo we could instead only list some
requirements (like top level bundles) and resolve the rest.

I have prototyped such a packaging using bndtools as it already has the
OBR resolving:
https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
The creation of the repository went quite smoothly as eclipse m2e
provides nice tooling to define the pom and the necessary excludes.
Defining the requirements in bndtools also works really nicely as they
got a good UI for it.

Unfortunately bndtools does not know about features. So while this
approach is better than defining each single bundle of an application it
is not really better than the karaf approach were you define most of
your deps as features. So this would only be really useable once we have
both features and a backing obr repo.

Another thing that I found bndtools is lacking is all the preparation
karaf already has for some of the more complicated features like
tansactions and cxf. There it is not enough to just install bundles.
You also have to fiddle with the boot path for transactions and some
java se spec nightmares in the case of cxf. So these are the regions
where karaf really helps.

Christian

On 18.04.2016 07:24, Brad Johnson wrote:

> Christian,
>
> I just re-read your section on the static profiles.  That makes
> sense.  I could see this becoming like puppet/chef sort of recipes or
> even like the way Docker allows building up of images.  Not that I
> know those that well.  I use features all the time of course but see
> huge advantage to making that a bundle time set of static steps
> instead of the current runtime usage.  Not that the current use is
> bad, mind you, its context is different and slimming down the karaf
> core by eliminating that management overhead would be good for
> something like karaf-boot.  In the current monolithic enterprise
> environment it makes sense to have stacks of features available to
> load when necessary.  The static mechanism would get rid of that. But
> it would also permit building up a centralized registry or library of
> features that one could leverage by adding into a boot recipe of some
> sort.   Maybe we'd get the quick flexibility for creating projects
> that archetypes always seemed to promise but never quit seemed to manage.
>
> One item that has always bothered me about features is they are
> orthogonal to but replicate much of what goes into a POM.   One ends
> up with two sets of dependency management mechanisms that have never
> really dovetailed.  It would be nice if we had a Maven plugin that
> could look at the dependencies in a POM and create a static feature
> profile or at least give a good guess at what they should be while
> allowing for some tweaking.  Perhaps since the karaf-boot environment
> is static and doesn't rely on or expect another environment to provide
> dependencies that would be easier to accomplish.
>
> Funny how Moore's law took a sideways turn on us.  Now we don't have
> the ever increasing clock speed but we have cores coming out our ears
> and RAM and disk space in abundance.  A little fatness in our
> deployments is an acceptable trade off now.
>
> Brad
>

--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
I don't know if it is a big deal but here's an example of a current project I work on. In a static environment much of that could be generated I'd think.  At least the parts that are straight bundles.  And really, in a static include the feature tags themselves could be pulled in and expanded to bundle statements as well since in the end we aren't leveraging features from an installed karaf anymore.  Those features/profiles would act more like recipes for the creation of the karaf-boot itself.

If one went with a convention over configuration approach as well then one could always specify a configuration.cfg in the root of the resources directory and have that automatically converted into a cfg file in the bundle that matched the PID.  The whole configfile section could be eliminated then.  If a configuration.cfg file is found in the resource directory then the static compilation could turn that into the correct PID.cfg and install it during bundle creation into the etc directory for boot up.

None of that is huge deal but it would make life a bit more convenient and since one is now working from a static point of view it seems it would be much easier to accomplish.

<?xml version="1.0" encoding="UTF-8"?>

  <!-- The repositories of the Middleware -->
  <repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>
  <repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>

  <feature name="payment-hub-gateway-prerequisites" version="${project.version}">

    <configfile finalname="/etc/com.foo.orchestration.cfg" override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
    <configfile finalname="/etc/com.foo.connector.litle.cfg" override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
    <configfile finalname="/etc/com.foo.gateway.cfg" override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
    <configfile finalname="/etc/com.foo.batch.cfg" override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
    <configfile finalname="/etc/com.foo.bar.cfg" override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
    <configfile finalname="/etc/com.foo.services.cfg" override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>

    <feature>camel-core</feature>
    <feature>camel-blueprint</feature>
    <feature>camel-cxf</feature>
    <feature>camel-jackson</feature>
    <feature>camel-xmljson</feature>
    <feature>camel-beanio</feature>
    <feature>camel-xstream</feature>
    <feature>camel-dozer</feature>
    <feature>camel-velocity</feature>
    <feature>cxf-ws-security</feature>
    
  <bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
    <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>
    <bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>
    <bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>
    <bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
    <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
 
  </feature>

  <feature name="foo-hub-gateway" version="${project.version}">

    <bundle>mvn:com.foo/models/${project.version}</bundle>
etc. etc.



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider <[hidden email]> wrote:
I am not very familiar with the concept of profiles till now. Guillaume explained it a bit to me but honestly I hope we do not need it too often. An OSGi application should be mainly built out of bundles and profiles concentrate more on the non OSGi static files in karaf.

The main part of the packaging in karaf boot is to resolve features at build time. It uses the static profile but nothing else from the profiles concept. This part already works. I have created such a packaging for the tasklist-ds example:
https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

The second part. The karaf boot development model is still in the making but you can already use the packaging if you want.

About features and poms you are right. You have to maintain two sets of dependencies but this is just a result of the fact that OSGi resolves differently from maven. So this never will completely go away. What we could do with a pom is create an obr repository. This could then be used to back a feature repo. So currently in a feature we have to list all the bundles, with a backing obr repo we could instead only list some requirements (like top level bundles) and resolve the rest.

I have prototyped such a packaging using bndtools as it already has the OBR resolving:
https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
The creation of the repository went quite smoothly as eclipse m2e provides nice tooling to define the pom and the necessary excludes.
Defining the requirements in bndtools also works really nicely as they got a good UI for it.

Unfortunately bndtools does not know about features. So while this approach is better than defining each single bundle of an application it is not really better than the karaf approach were you define most of your deps as features. So this would only be really useable once we have both features and a backing obr repo.

Another thing that I found bndtools is lacking is all the preparation karaf already has for some of the more complicated features like tansactions and cxf. There it is not enough to just install bundles.
You also have to fiddle with the boot path for transactions and some java se spec nightmares in the case of cxf. So these are the regions where karaf really helps.

Christian


On 18.04.2016 07:24, Brad Johnson wrote:
Christian,

I just re-read your section on the static profiles.  That makes sense.  I could see this becoming like puppet/chef sort of recipes or even like the way Docker allows building up of images.  Not that I know those that well.  I use features all the time of course but see huge advantage to making that a bundle time set of static steps instead of the current runtime usage.  Not that the current use is bad, mind you, its context is different and slimming down the karaf core by eliminating that management overhead would be good for something like karaf-boot.  In the current monolithic enterprise environment it makes sense to have stacks of features available to load when necessary.  The static mechanism would get rid of that. But it would also permit building up a centralized registry or library of features that one could leverage by adding into a boot recipe of some sort.   Maybe we'd get the quick flexibility for creating projects that archetypes always seemed to promise but never quit seemed to manage.

One item that has always bothered me about features is they are orthogonal to but replicate much of what goes into a POM.   One ends up with two sets of dependency management mechanisms that have never really dovetailed.  It would be nice if we had a Maven plugin that could look at the dependencies in a POM and create a static feature profile or at least give a good guess at what they should be while allowing for some tweaking.  Perhaps since the karaf-boot environment is static and doesn't rely on or expect another environment to provide dependencies that would be easier to accomplish.

Funny how Moore's law took a sideways turn on us.  Now we don't have the ever increasing clock speed but we have cores coming out our ears and RAM and disk space in abundance.  A little fatness in our deployments is an acceptable trade off now.

Brad


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com


Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Jean-Baptiste Onofré
Starting from features, you can create a full static custom
distribution, not actually really using features but startup bundles.

Regards
JB

On 04/18/2016 05:47 PM, Brad Johnson wrote:

> I don't know if it is a big deal but here's an example of a current
> project I work on. In a static environment much of that could be
> generated I'd think.  At least the parts that are straight bundles.  And
> really, in a static include the feature tags themselves could be pulled
> in and expanded to bundle statements as well since in the end we aren't
> leveraging features from an installed karaf anymore.  Those
> features/profiles would act more like recipes for the creation of the
> karaf-boot itself.
>
> If one went with a convention over configuration approach as well then
> one could always specify a configuration.cfg in the root of the
> resources directory and have that automatically converted into a cfg
> file in the bundle that matched the PID.  The whole configfile section
> could be eliminated then.  If a configuration.cfg file is found in the
> resource directory then the static compilation could turn that into the
> correct PID.cfg and install it during bundle creation into the etc
> directory for boot up.
>
> None of that is huge deal but it would make life a bit more convenient
> and since one is now working from a static point of view it seems it
> would be much easier to accomplish.
>
> <?xml version="1.0" encoding="UTF-8"?>
> <features name="${project.artifactId}-${project.version}"
> xmlns="http://karaf.apache.org/xmlns/features/v1.0.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.0.0
> http://karaf.apache.org/xmlns/features/v1.0.0">
>
>    <!-- The repositories of the Middleware -->
>
> <repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>
>
> <repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>
>
>    <feature name="payment-hub-gateway-prerequisites"
> version="${project.version}">
>
>      <configfile finalname="/etc/com.foo.orchestration.cfg"
> override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
>      <configfile finalname="/etc/com.foo.connector.litle.cfg"
> override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
>      <configfile finalname="/etc/com.foo.gateway.cfg"
> override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
>      <configfile finalname="/etc/com.foo.batch.cfg"
> override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
>      <configfile finalname="/etc/com.foo.bar.cfg"
> override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
>      <configfile finalname="/etc/com.foo.services.cfg"
> override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>
>
>      <feature>camel-core</feature>
>      <feature>camel-blueprint</feature>
>      <feature>camel-cxf</feature>
>      <feature>camel-jackson</feature>
>      <feature>camel-xmljson</feature>
>      <feature>camel-beanio</feature>
>      <feature>camel-xstream</feature>
>      <feature>camel-dozer</feature>
>      <feature>camel-velocity</feature>
>      <feature>cxf-ws-security</feature>
> <bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
>      <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>
>
> <bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>
>
> <bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>
>
> <bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
>      <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
>    </feature>
>
>    <feature name="foo-hub-gateway" version="${project.version}">
>
>      <bundle>mvn:com.foo/models/${project.version}</bundle>
> etc. etc.
>
>
>
> On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     I am not very familiar with the concept of profiles till now.
>     Guillaume explained it a bit to me but honestly I hope we do not
>     need it too often. An OSGi application should be mainly built out of
>     bundles and profiles concentrate more on the non OSGi static files
>     in karaf.
>
>     The main part of the packaging in karaf boot is to resolve features
>     at build time. It uses the static profile but nothing else from the
>     profiles concept. This part already works. I have created such a
>     packaging for the tasklist-ds example:
>     https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app
>
>     The second part. The karaf boot development model is still in the
>     making but you can already use the packaging if you want.
>
>     About features and poms you are right. You have to maintain two sets
>     of dependencies but this is just a result of the fact that OSGi
>     resolves differently from maven. So this never will completely go
>     away. What we could do with a pom is create an obr repository. This
>     could then be used to back a feature repo. So currently in a feature
>     we have to list all the bundles, with a backing obr repo we could
>     instead only list some requirements (like top level bundles) and
>     resolve the rest.
>
>     I have prototyped such a packaging using bndtools as it already has
>     the OBR resolving:
>     https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
>     The creation of the repository went quite smoothly as eclipse m2e
>     provides nice tooling to define the pom and the necessary excludes.
>     Defining the requirements in bndtools also works really nicely as
>     they got a good UI for it.
>
>     Unfortunately bndtools does not know about features. So while this
>     approach is better than defining each single bundle of an
>     application it is not really better than the karaf approach were you
>     define most of your deps as features. So this would only be really
>     useable once we have both features and a backing obr repo.
>
>     Another thing that I found bndtools is lacking is all the
>     preparation karaf already has for some of the more complicated
>     features like tansactions and cxf. There it is not enough to just
>     install bundles.
>     You also have to fiddle with the boot path for transactions and some
>     java se spec nightmares in the case of cxf. So these are the regions
>     where karaf really helps.
>
>     Christian
>
>     On 18.04.2016 07:24, Brad Johnson wrote:
>
>         Christian,
>
>         I just re-read your section on the static profiles.  That makes
>         sense.  I could see this becoming like puppet/chef sort of
>         recipes or even like the way Docker allows building up of
>         images.  Not that I know those that well.  I use features all
>         the time of course but see huge advantage to making that a
>         bundle time set of static steps instead of the current runtime
>         usage.  Not that the current use is bad, mind you, its context
>         is different and slimming down the karaf core by eliminating
>         that management overhead would be good for something like
>         karaf-boot.  In the current monolithic enterprise environment it
>         makes sense to have stacks of features available to load when
>         necessary.  The static mechanism would get rid of that. But it
>         would also permit building up a centralized registry or library
>         of features that one could leverage by adding into a boot recipe
>         of some sort.   Maybe we'd get the quick flexibility for
>         creating projects that archetypes always seemed to promise but
>         never quit seemed to manage.
>
>         One item that has always bothered me about features is they are
>         orthogonal to but replicate much of what goes into a POM.   One
>         ends up with two sets of dependency management mechanisms that
>         have never really dovetailed.  It would be nice if we had a
>         Maven plugin that could look at the dependencies in a POM and
>         create a static feature profile or at least give a good guess at
>         what they should be while allowing for some tweaking.  Perhaps
>         since the karaf-boot environment is static and doesn't rely on
>         or expect another environment to provide dependencies that would
>         be easier to accomplish.
>
>         Funny how Moore's law took a sideways turn on us.  Now we don't
>         have the ever increasing clock speed but we have cores coming
>         out our ears and RAM and disk space in abundance.  A little
>         fatness in our deployments is an acceptable trade off now.
>
>         Brad
>
>
>     --
>     Christian Schneider
>     http://www.liquid-reality.de
>
>     Open Source Architect
>     http://www.talend.com
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
In reply to this post by Christian Schneider
Christian,

I think your idea of the OBR repository for features and profiles is good.  I think when I start using karaf-boot what I'd certainly want is a way to specify features almost like recipes.  Those would obviously have to be inlined in the bundles features file as their true mvn:group/artifact/version form since everything at that point is static.  We needn't have a look up mechanism at runtime obviously. In similar fashion one could have a plugin look at the POM and dependencies specified as being required for running the bundle and have those installed. Unfortunately I don't know how Karaf-boot is going to install those.  In lib? endorsed?  I'd think there would be a number of possible ways to add those and the static nature will simplify much of it.

Brad



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider <[hidden email]> wrote:
I am not very familiar with the concept of profiles till now. Guillaume explained it a bit to me but honestly I hope we do not need it too often. An OSGi application should be mainly built out of bundles and profiles concentrate more on the non OSGi static files in karaf.

The main part of the packaging in karaf boot is to resolve features at build time. It uses the static profile but nothing else from the profiles concept. This part already works. I have created such a packaging for the tasklist-ds example:
https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

The second part. The karaf boot development model is still in the making but you can already use the packaging if you want.

About features and poms you are right. You have to maintain two sets of dependencies but this is just a result of the fact that OSGi resolves differently from maven. So this never will completely go away. What we could do with a pom is create an obr repository. This could then be used to back a feature repo. So currently in a feature we have to list all the bundles, with a backing obr repo we could instead only list some requirements (like top level bundles) and resolve the rest.

I have prototyped such a packaging using bndtools as it already has the OBR resolving:
https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
The creation of the repository went quite smoothly as eclipse m2e provides nice tooling to define the pom and the necessary excludes.
Defining the requirements in bndtools also works really nicely as they got a good UI for it.

Unfortunately bndtools does not know about features. So while this approach is better than defining each single bundle of an application it is not really better than the karaf approach were you define most of your deps as features. So this would only be really useable once we have both features and a backing obr repo.

Another thing that I found bndtools is lacking is all the preparation karaf already has for some of the more complicated features like tansactions and cxf. There it is not enough to just install bundles.
You also have to fiddle with the boot path for transactions and some java se spec nightmares in the case of cxf. So these are the regions where karaf really helps.

Christian


On 18.04.2016 07:24, Brad Johnson wrote:
Christian,

I just re-read your section on the static profiles.  That makes sense.  I could see this becoming like puppet/chef sort of recipes or even like the way Docker allows building up of images.  Not that I know those that well.  I use features all the time of course but see huge advantage to making that a bundle time set of static steps instead of the current runtime usage.  Not that the current use is bad, mind you, its context is different and slimming down the karaf core by eliminating that management overhead would be good for something like karaf-boot.  In the current monolithic enterprise environment it makes sense to have stacks of features available to load when necessary.  The static mechanism would get rid of that. But it would also permit building up a centralized registry or library of features that one could leverage by adding into a boot recipe of some sort.   Maybe we'd get the quick flexibility for creating projects that archetypes always seemed to promise but never quit seemed to manage.

One item that has always bothered me about features is they are orthogonal to but replicate much of what goes into a POM.   One ends up with two sets of dependency management mechanisms that have never really dovetailed.  It would be nice if we had a Maven plugin that could look at the dependencies in a POM and create a static feature profile or at least give a good guess at what they should be while allowing for some tweaking.  Perhaps since the karaf-boot environment is static and doesn't rely on or expect another environment to provide dependencies that would be easier to accomplish.

Funny how Moore's law took a sideways turn on us.  Now we don't have the ever increasing clock speed but we have cores coming out our ears and RAM and disk space in abundance.  A little fatness in our deployments is an acceptable trade off now.

Brad


--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com


Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
In reply to this post by Jean-Baptiste Onofré
Jean-Baptiste,

So would something like a camel-core feature then be converted into a static bundle with all its dependencies for bootstrap?  It seems that what one could do is define the features and the read the features into a set at bundle time to eliminate any overlap and then create the list of required mvn:group/artifact statements in the new bundle.

But that also seems a prime candidate for leveraging Maven to create that same thing.  It has all the information required to create that set of required bundles for bootstrap sans features.  But karaf would have to be made aware of them somehow. Would these be specified in the startup.properties and stored in the lib? 

Sorry, I should probably just wait and see as I imagine it gets a bit irritating to go over stuff you've already thought out.  I'm pretty stoked about the possibility though so my mind naturally races on to how I'd like to see it work and how it could make my life easier.

Brad



On Mon, Apr 18, 2016 at 10:52 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
Starting from features, you can create a full static custom distribution, not actually really using features but startup bundles.

Regards
JB


On 04/18/2016 05:47 PM, Brad Johnson wrote:
I don't know if it is a big deal but here's an example of a current
project I work on. In a static environment much of that could be
generated I'd think.  At least the parts that are straight bundles.  And
really, in a static include the feature tags themselves could be pulled
in and expanded to bundle statements as well since in the end we aren't
leveraging features from an installed karaf anymore.  Those
features/profiles would act more like recipes for the creation of the
karaf-boot itself.

If one went with a convention over configuration approach as well then
one could always specify a configuration.cfg in the root of the
resources directory and have that automatically converted into a cfg
file in the bundle that matched the PID.  The whole configfile section
could be eliminated then.  If a configuration.cfg file is found in the
resource directory then the static compilation could turn that into the
correct PID.cfg and install it during bundle creation into the etc
directory for boot up.

None of that is huge deal but it would make life a bit more convenient
and since one is now working from a static point of view it seems it
would be much easier to accomplish.

<?xml version="1.0" encoding="UTF-8"?>
<features name="${project.artifactId}-${project.version}"
xmlns="http://karaf.apache.org/xmlns/features/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.0.0
http://karaf.apache.org/xmlns/features/v1.0.0">

   <!-- The repositories of the Middleware -->

<repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>

<repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>

   <feature name="payment-hub-gateway-prerequisites"
version="${project.version}">

     <configfile finalname="/etc/com.foo.orchestration.cfg"
override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.connector.litle.cfg"
override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.gateway.cfg"
override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.batch.cfg"
override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.bar.cfg"
override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.services.cfg"
override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>

     <feature>camel-core</feature>
     <feature>camel-blueprint</feature>
     <feature>camel-cxf</feature>
     <feature>camel-jackson</feature>
     <feature>camel-xmljson</feature>
     <feature>camel-beanio</feature>
     <feature>camel-xstream</feature>
     <feature>camel-dozer</feature>
     <feature>camel-velocity</feature>
     <feature>cxf-ws-security</feature>
<bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
     <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>

<bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
     <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
   </feature>

   <feature name="foo-hub-gateway" version="${project.version}">

     <bundle>mvn:com.foo/models/${project.version}</bundle>
etc. etc.



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I am not very familiar with the concept of profiles till now.
    Guillaume explained it a bit to me but honestly I hope we do not
    need it too often. An OSGi application should be mainly built out of
    bundles and profiles concentrate more on the non OSGi static files
    in karaf.

    The main part of the packaging in karaf boot is to resolve features
    at build time. It uses the static profile but nothing else from the
    profiles concept. This part already works. I have created such a
    packaging for the tasklist-ds example:
    https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

    The second part. The karaf boot development model is still in the
    making but you can already use the packaging if you want.

    About features and poms you are right. You have to maintain two sets
    of dependencies but this is just a result of the fact that OSGi
    resolves differently from maven. So this never will completely go
    away. What we could do with a pom is create an obr repository. This
    could then be used to back a feature repo. So currently in a feature
    we have to list all the bundles, with a backing obr repo we could
    instead only list some requirements (like top level bundles) and
    resolve the rest.

    I have prototyped such a packaging using bndtools as it already has
    the OBR resolving:
    https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
    The creation of the repository went quite smoothly as eclipse m2e
    provides nice tooling to define the pom and the necessary excludes.
    Defining the requirements in bndtools also works really nicely as
    they got a good UI for it.

    Unfortunately bndtools does not know about features. So while this
    approach is better than defining each single bundle of an
    application it is not really better than the karaf approach were you
    define most of your deps as features. So this would only be really
    useable once we have both features and a backing obr repo.

    Another thing that I found bndtools is lacking is all the
    preparation karaf already has for some of the more complicated
    features like tansactions and cxf. There it is not enough to just
    install bundles.
    You also have to fiddle with the boot path for transactions and some
    java se spec nightmares in the case of cxf. So these are the regions
    where karaf really helps.

    Christian

    On 18.04.2016 07:24, Brad Johnson wrote:

        Christian,

        I just re-read your section on the static profiles.  That makes
        sense.  I could see this becoming like puppet/chef sort of
        recipes or even like the way Docker allows building up of
        images.  Not that I know those that well.  I use features all
        the time of course but see huge advantage to making that a
        bundle time set of static steps instead of the current runtime
        usage.  Not that the current use is bad, mind you, its context
        is different and slimming down the karaf core by eliminating
        that management overhead would be good for something like
        karaf-boot.  In the current monolithic enterprise environment it
        makes sense to have stacks of features available to load when
        necessary.  The static mechanism would get rid of that. But it
        would also permit building up a centralized registry or library
        of features that one could leverage by adding into a boot recipe
        of some sort.   Maybe we'd get the quick flexibility for
        creating projects that archetypes always seemed to promise but
        never quit seemed to manage.

        One item that has always bothered me about features is they are
        orthogonal to but replicate much of what goes into a POM.   One
        ends up with two sets of dependency management mechanisms that
        have never really dovetailed.  It would be nice if we had a
        Maven plugin that could look at the dependencies in a POM and
        create a static feature profile or at least give a good guess at
        what they should be while allowing for some tweaking.  Perhaps
        since the karaf-boot environment is static and doesn't rely on
        or expect another environment to provide dependencies that would
        be easier to accomplish.

        Funny how Moore's law took a sideways turn on us.  Now we don't
        have the ever increasing clock speed but we have cores coming
        out our ears and RAM and disk space in abundance.  A little
        fatness in our deployments is an acceptable trade off now.

        Brad


    --
    Christian Schneider
    http://www.liquid-reality.de

    Open Source Architect
    http://www.talend.com



Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Guillaume Nodet-2


2016-04-18 18:02 GMT+02:00 Brad Johnson <[hidden email]>:
Jean-Baptiste,

So would something like a camel-core feature then be converted into a static bundle with all its dependencies for bootstrap?  It seems that what one could do is define the features and the read the features into a set at bundle time to eliminate any overlap and then create the list of required mvn:group/artifact statements in the new bundle.

Well, that's actually what the karaf plugin does when generating the assembly.
Though it does the same thing than the OSGi resolver would do at runtime, and handles additional things such as configuration, start levels, libraries, etc...
 

But that also seems a prime candidate for leveraging Maven to create that same thing.  It has all the information required to create that set of required bundles for bootstrap sans features.  But karaf would have to be made aware of them somehow. Would these be specified in the startup.properties and stored in the lib? 

Sorry, I should probably just wait and see as I imagine it gets a bit irritating to go over stuff you've already thought out.  I'm pretty stoked about the possibility though so my mind naturally races on to how I'd like to see it work and how it could make my life easier.

Brad



On Mon, Apr 18, 2016 at 10:52 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
Starting from features, you can create a full static custom distribution, not actually really using features but startup bundles.

Regards
JB


On 04/18/2016 05:47 PM, Brad Johnson wrote:
I don't know if it is a big deal but here's an example of a current
project I work on. In a static environment much of that could be
generated I'd think.  At least the parts that are straight bundles.  And
really, in a static include the feature tags themselves could be pulled
in and expanded to bundle statements as well since in the end we aren't
leveraging features from an installed karaf anymore.  Those
features/profiles would act more like recipes for the creation of the
karaf-boot itself.

If one went with a convention over configuration approach as well then
one could always specify a configuration.cfg in the root of the
resources directory and have that automatically converted into a cfg
file in the bundle that matched the PID.  The whole configfile section
could be eliminated then.  If a configuration.cfg file is found in the
resource directory then the static compilation could turn that into the
correct PID.cfg and install it during bundle creation into the etc
directory for boot up.

None of that is huge deal but it would make life a bit more convenient
and since one is now working from a static point of view it seems it
would be much easier to accomplish.

<?xml version="1.0" encoding="UTF-8"?>
<features name="${project.artifactId}-${project.version}"
xmlns="http://karaf.apache.org/xmlns/features/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.0.0
http://karaf.apache.org/xmlns/features/v1.0.0">

   <!-- The repositories of the Middleware -->

<repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>

<repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>

   <feature name="payment-hub-gateway-prerequisites"
version="${project.version}">

     <configfile finalname="/etc/com.foo.orchestration.cfg"
override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.connector.litle.cfg"
override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.gateway.cfg"
override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.batch.cfg"
override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.bar.cfg"
override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.services.cfg"
override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>

     <feature>camel-core</feature>
     <feature>camel-blueprint</feature>
     <feature>camel-cxf</feature>
     <feature>camel-jackson</feature>
     <feature>camel-xmljson</feature>
     <feature>camel-beanio</feature>
     <feature>camel-xstream</feature>
     <feature>camel-dozer</feature>
     <feature>camel-velocity</feature>
     <feature>cxf-ws-security</feature>
<bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
     <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>

<bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
     <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
   </feature>

   <feature name="foo-hub-gateway" version="${project.version}">

     <bundle>mvn:com.foo/models/${project.version}</bundle>
etc. etc.



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I am not very familiar with the concept of profiles till now.
    Guillaume explained it a bit to me but honestly I hope we do not
    need it too often. An OSGi application should be mainly built out of
    bundles and profiles concentrate more on the non OSGi static files
    in karaf.

    The main part of the packaging in karaf boot is to resolve features
    at build time. It uses the static profile but nothing else from the
    profiles concept. This part already works. I have created such a
    packaging for the tasklist-ds example:
    https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

    The second part. The karaf boot development model is still in the
    making but you can already use the packaging if you want.

    About features and poms you are right. You have to maintain two sets
    of dependencies but this is just a result of the fact that OSGi
    resolves differently from maven. So this never will completely go
    away. What we could do with a pom is create an obr repository. This
    could then be used to back a feature repo. So currently in a feature
    we have to list all the bundles, with a backing obr repo we could
    instead only list some requirements (like top level bundles) and
    resolve the rest.

    I have prototyped such a packaging using bndtools as it already has
    the OBR resolving:
    https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
    The creation of the repository went quite smoothly as eclipse m2e
    provides nice tooling to define the pom and the necessary excludes.
    Defining the requirements in bndtools also works really nicely as
    they got a good UI for it.

    Unfortunately bndtools does not know about features. So while this
    approach is better than defining each single bundle of an
    application it is not really better than the karaf approach were you
    define most of your deps as features. So this would only be really
    useable once we have both features and a backing obr repo.

    Another thing that I found bndtools is lacking is all the
    preparation karaf already has for some of the more complicated
    features like tansactions and cxf. There it is not enough to just
    install bundles.
    You also have to fiddle with the boot path for transactions and some
    java se spec nightmares in the case of cxf. So these are the regions
    where karaf really helps.

    Christian

    On 18.04.2016 07:24, Brad Johnson wrote:

        Christian,

        I just re-read your section on the static profiles.  That makes
        sense.  I could see this becoming like puppet/chef sort of
        recipes or even like the way Docker allows building up of
        images.  Not that I know those that well.  I use features all
        the time of course but see huge advantage to making that a
        bundle time set of static steps instead of the current runtime
        usage.  Not that the current use is bad, mind you, its context
        is different and slimming down the karaf core by eliminating
        that management overhead would be good for something like
        karaf-boot.  In the current monolithic enterprise environment it
        makes sense to have stacks of features available to load when
        necessary.  The static mechanism would get rid of that. But it
        would also permit building up a centralized registry or library
        of features that one could leverage by adding into a boot recipe
        of some sort.   Maybe we'd get the quick flexibility for
        creating projects that archetypes always seemed to promise but
        never quit seemed to manage.

        One item that has always bothered me about features is they are
        orthogonal to but replicate much of what goes into a POM.   One
        ends up with two sets of dependency management mechanisms that
        have never really dovetailed.  It would be nice if we had a
        Maven plugin that could look at the dependencies in a POM and
        create a static feature profile or at least give a good guess at
        what they should be while allowing for some tweaking.  Perhaps
        since the karaf-boot environment is static and doesn't rely on
        or expect another environment to provide dependencies that would
        be easier to accomplish.

        Funny how Moore's law took a sideways turn on us.  Now we don't
        have the ever increasing clock speed but we have cores coming
        out our ears and RAM and disk space in abundance.  A little
        fatness in our deployments is an acceptable trade off now.

        Brad


    --
    Christian Schneider
    http://www.liquid-reality.de

    Open Source Architect
    http://www.talend.com






--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration


Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Brad Johnson
Sounds pretty spectacular.  Can't wait to see it.

Brad

On Mon, Apr 18, 2016 at 11:13 AM, Guillaume Nodet <[hidden email]> wrote:


2016-04-18 18:02 GMT+02:00 Brad Johnson <[hidden email]>:
Jean-Baptiste,

So would something like a camel-core feature then be converted into a static bundle with all its dependencies for bootstrap?  It seems that what one could do is define the features and the read the features into a set at bundle time to eliminate any overlap and then create the list of required mvn:group/artifact statements in the new bundle.

Well, that's actually what the karaf plugin does when generating the assembly.
Though it does the same thing than the OSGi resolver would do at runtime, and handles additional things such as configuration, start levels, libraries, etc...
 

But that also seems a prime candidate for leveraging Maven to create that same thing.  It has all the information required to create that set of required bundles for bootstrap sans features.  But karaf would have to be made aware of them somehow. Would these be specified in the startup.properties and stored in the lib? 

Sorry, I should probably just wait and see as I imagine it gets a bit irritating to go over stuff you've already thought out.  I'm pretty stoked about the possibility though so my mind naturally races on to how I'd like to see it work and how it could make my life easier.

Brad



On Mon, Apr 18, 2016 at 10:52 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
Starting from features, you can create a full static custom distribution, not actually really using features but startup bundles.

Regards
JB


On 04/18/2016 05:47 PM, Brad Johnson wrote:
I don't know if it is a big deal but here's an example of a current
project I work on. In a static environment much of that could be
generated I'd think.  At least the parts that are straight bundles.  And
really, in a static include the feature tags themselves could be pulled
in and expanded to bundle statements as well since in the end we aren't
leveraging features from an installed karaf anymore.  Those
features/profiles would act more like recipes for the creation of the
karaf-boot itself.

If one went with a convention over configuration approach as well then
one could always specify a configuration.cfg in the root of the
resources directory and have that automatically converted into a cfg
file in the bundle that matched the PID.  The whole configfile section
could be eliminated then.  If a configuration.cfg file is found in the
resource directory then the static compilation could turn that into the
correct PID.cfg and install it during bundle creation into the etc
directory for boot up.

None of that is huge deal but it would make life a bit more convenient
and since one is now working from a static point of view it seems it
would be much easier to accomplish.

<?xml version="1.0" encoding="UTF-8"?>
<features name="${project.artifactId}-${project.version}"
xmlns="http://karaf.apache.org/xmlns/features/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.0.0
http://karaf.apache.org/xmlns/features/v1.0.0">

   <!-- The repositories of the Middleware -->

<repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>

<repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>

   <feature name="payment-hub-gateway-prerequisites"
version="${project.version}">

     <configfile finalname="/etc/com.foo.orchestration.cfg"
override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.connector.litle.cfg"
override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.gateway.cfg"
override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.batch.cfg"
override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.bar.cfg"
override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.services.cfg"
override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>

     <feature>camel-core</feature>
     <feature>camel-blueprint</feature>
     <feature>camel-cxf</feature>
     <feature>camel-jackson</feature>
     <feature>camel-xmljson</feature>
     <feature>camel-beanio</feature>
     <feature>camel-xstream</feature>
     <feature>camel-dozer</feature>
     <feature>camel-velocity</feature>
     <feature>cxf-ws-security</feature>
<bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
     <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>

<bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
     <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
   </feature>

   <feature name="foo-hub-gateway" version="${project.version}">

     <bundle>mvn:com.foo/models/${project.version}</bundle>
etc. etc.



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I am not very familiar with the concept of profiles till now.
    Guillaume explained it a bit to me but honestly I hope we do not
    need it too often. An OSGi application should be mainly built out of
    bundles and profiles concentrate more on the non OSGi static files
    in karaf.

    The main part of the packaging in karaf boot is to resolve features
    at build time. It uses the static profile but nothing else from the
    profiles concept. This part already works. I have created such a
    packaging for the tasklist-ds example:
    https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

    The second part. The karaf boot development model is still in the
    making but you can already use the packaging if you want.

    About features and poms you are right. You have to maintain two sets
    of dependencies but this is just a result of the fact that OSGi
    resolves differently from maven. So this never will completely go
    away. What we could do with a pom is create an obr repository. This
    could then be used to back a feature repo. So currently in a feature
    we have to list all the bundles, with a backing obr repo we could
    instead only list some requirements (like top level bundles) and
    resolve the rest.

    I have prototyped such a packaging using bndtools as it already has
    the OBR resolving:
    https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
    The creation of the repository went quite smoothly as eclipse m2e
    provides nice tooling to define the pom and the necessary excludes.
    Defining the requirements in bndtools also works really nicely as
    they got a good UI for it.

    Unfortunately bndtools does not know about features. So while this
    approach is better than defining each single bundle of an
    application it is not really better than the karaf approach were you
    define most of your deps as features. So this would only be really
    useable once we have both features and a backing obr repo.

    Another thing that I found bndtools is lacking is all the
    preparation karaf already has for some of the more complicated
    features like tansactions and cxf. There it is not enough to just
    install bundles.
    You also have to fiddle with the boot path for transactions and some
    java se spec nightmares in the case of cxf. So these are the regions
    where karaf really helps.

    Christian

    On 18.04.2016 07:24, Brad Johnson wrote:

        Christian,

        I just re-read your section on the static profiles.  That makes
        sense.  I could see this becoming like puppet/chef sort of
        recipes or even like the way Docker allows building up of
        images.  Not that I know those that well.  I use features all
        the time of course but see huge advantage to making that a
        bundle time set of static steps instead of the current runtime
        usage.  Not that the current use is bad, mind you, its context
        is different and slimming down the karaf core by eliminating
        that management overhead would be good for something like
        karaf-boot.  In the current monolithic enterprise environment it
        makes sense to have stacks of features available to load when
        necessary.  The static mechanism would get rid of that. But it
        would also permit building up a centralized registry or library
        of features that one could leverage by adding into a boot recipe
        of some sort.   Maybe we'd get the quick flexibility for
        creating projects that archetypes always seemed to promise but
        never quit seemed to manage.

        One item that has always bothered me about features is they are
        orthogonal to but replicate much of what goes into a POM.   One
        ends up with two sets of dependency management mechanisms that
        have never really dovetailed.  It would be nice if we had a
        Maven plugin that could look at the dependencies in a POM and
        create a static feature profile or at least give a good guess at
        what they should be while allowing for some tweaking.  Perhaps
        since the karaf-boot environment is static and doesn't rely on
        or expect another environment to provide dependencies that would
        be easier to accomplish.

        Funny how Moore's law took a sideways turn on us.  Now we don't
        have the ever increasing clock speed but we have cores coming
        out our ears and RAM and disk space in abundance.  A little
        fatness in our deployments is an acceptable trade off now.

        Brad


    --
    Christian Schneider
    http://www.liquid-reality.de

    Open Source Architect
    http://www.talend.com






--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration



Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Christian Schneider
It is already there. Give this example a spin:
https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

Should be quite easy to adapt this to your own project.

Christian

On 18.04.2016 18:19, Brad Johnson wrote:
Sounds pretty spectacular.  Can't wait to see it.

Brad

On Mon, Apr 18, 2016 at 11:13 AM, Guillaume Nodet <[hidden email]> wrote:


2016-04-18 18:02 GMT+02:00 Brad Johnson <[hidden email]>:
Jean-Baptiste,

So would something like a camel-core feature then be converted into a static bundle with all its dependencies for bootstrap?  It seems that what one could do is define the features and the read the features into a set at bundle time to eliminate any overlap and then create the list of required mvn:group/artifact statements in the new bundle.

Well, that's actually what the karaf plugin does when generating the assembly.
Though it does the same thing than the OSGi resolver would do at runtime, and handles additional things such as configuration, start levels, libraries, etc...
 

But that also seems a prime candidate for leveraging Maven to create that same thing.  It has all the information required to create that set of required bundles for bootstrap sans features.  But karaf would have to be made aware of them somehow. Would these be specified in the startup.properties and stored in the lib? 

Sorry, I should probably just wait and see as I imagine it gets a bit irritating to go over stuff you've already thought out.  I'm pretty stoked about the possibility though so my mind naturally races on to how I'd like to see it work and how it could make my life easier.

Brad



On Mon, Apr 18, 2016 at 10:52 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
Starting from features, you can create a full static custom distribution, not actually really using features but startup bundles.

Regards
JB


On 04/18/2016 05:47 PM, Brad Johnson wrote:
I don't know if it is a big deal but here's an example of a current
project I work on. In a static environment much of that could be
generated I'd think.  At least the parts that are straight bundles.  And
really, in a static include the feature tags themselves could be pulled
in and expanded to bundle statements as well since in the end we aren't
leveraging features from an installed karaf anymore.  Those
features/profiles would act more like recipes for the creation of the
karaf-boot itself.

If one went with a convention over configuration approach as well then
one could always specify a configuration.cfg in the root of the
resources directory and have that automatically converted into a cfg
file in the bundle that matched the PID.  The whole configfile section
could be eliminated then.  If a configuration.cfg file is found in the
resource directory then the static compilation could turn that into the
correct PID.cfg and install it during bundle creation into the etc
directory for boot up.

None of that is huge deal but it would make life a bit more convenient
and since one is now working from a static point of view it seems it
would be much easier to accomplish.

<?xml version="1.0" encoding="UTF-8"?>
<features name="${project.artifactId}-${project.version}"
xmlns="http://karaf.apache.org/xmlns/features/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.0.0
http://karaf.apache.org/xmlns/features/v1.0.0">

   <!-- The repositories of the Middleware -->

<repository>mvn:org.apache.camel.karaf/apache-camel/${camel-version}/xml/features</repository>

<repository>mvn:org.apache.cxf.karaf/apache-cxf/${cxf-version}/xml/features</repository>

   <feature name="payment-hub-gateway-prerequisites"
version="${project.version}">

     <configfile finalname="/etc/com.foo.orchestration.cfg"
override="false">mvn:com.foo/orchestration/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.connector.litle.cfg"
override="false">mvn:com.foo/connector.litle/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.gateway.cfg"
override="false">mvn:com.foo/gateway/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.batch.cfg"
override="false">mvn:com.foo/batch/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.bar.cfg"
override="false">mvn:com.foo/bar/${project.version}/cfg/configuration</configfile>
     <configfile finalname="/etc/com.foo.services.cfg"
override="false">mvn:com.foo/services/${project.version}/cfg/configuration</configfile>

     <feature>camel-core</feature>
     <feature>camel-blueprint</feature>
     <feature>camel-cxf</feature>
     <feature>camel-jackson</feature>
     <feature>camel-xmljson</feature>
     <feature>camel-beanio</feature>
     <feature>camel-xstream</feature>
     <feature>camel-dozer</feature>
     <feature>camel-velocity</feature>
     <feature>cxf-ws-security</feature>
<bundle>mvn:org.apache.aries.blueprint/org.apache.aries.blueprint.core.compatibility/1.0.0</bundle>
     <bundle>mvn:org.codehaus.jettison/jettison/${jettison-version}</bundle>

<bundle>mvn:org.apache.commons/commons-lang3/${commons-lang3-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpcore/${httpcore-version}</bundle>

<bundle>wrap:mvn:org.apache.httpcomponents/httpclient/${httpclient-version}</bundle>
     <bundle>wrap:mvn:commons-io/commons-io/${commons-io-version}</bundle>
   </feature>

   <feature name="foo-hub-gateway" version="${project.version}">

     <bundle>mvn:com.foo/models/${project.version}</bundle>
etc. etc.



On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
<[hidden email] <mailto:[hidden email]>> wrote:

    I am not very familiar with the concept of profiles till now.
    Guillaume explained it a bit to me but honestly I hope we do not
    need it too often. An OSGi application should be mainly built out of
    bundles and profiles concentrate more on the non OSGi static files
    in karaf.

    The main part of the packaging in karaf boot is to resolve features
    at build time. It uses the static profile but nothing else from the
    profiles concept. This part already works. I have created such a
    packaging for the tasklist-ds example:
    https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app

    The second part. The karaf boot development model is still in the
    making but you can already use the packaging if you want.

    About features and poms you are right. You have to maintain two sets
    of dependencies but this is just a result of the fact that OSGi
    resolves differently from maven. So this never will completely go
    away. What we could do with a pom is create an obr repository. This
    could then be used to back a feature repo. So currently in a feature
    we have to list all the bundles, with a backing obr repo we could
    instead only list some requirements (like top level bundles) and
    resolve the rest.

    I have prototyped such a packaging using bndtools as it already has
    the OBR resolving:
    https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
    The creation of the repository went quite smoothly as eclipse m2e
    provides nice tooling to define the pom and the necessary excludes.
    Defining the requirements in bndtools also works really nicely as
    they got a good UI for it.

    Unfortunately bndtools does not know about features. So while this
    approach is better than defining each single bundle of an
    application it is not really better than the karaf approach were you
    define most of your deps as features. So this would only be really
    useable once we have both features and a backing obr repo.

    Another thing that I found bndtools is lacking is all the
    preparation karaf already has for some of the more complicated
    features like tansactions and cxf. There it is not enough to just
    install bundles.
    You also have to fiddle with the boot path for transactions and some
    java se spec nightmares in the case of cxf. So these are the regions
    where karaf really helps.

    Christian

    On 18.04.2016 07:24, Brad Johnson wrote:

        Christian,

        I just re-read your section on the static profiles.  That makes
        sense.  I could see this becoming like puppet/chef sort of
        recipes or even like the way Docker allows building up of
        images.  Not that I know those that well.  I use features all
        the time of course but see huge advantage to making that a
        bundle time set of static steps instead of the current runtime
        usage.  Not that the current use is bad, mind you, its context
        is different and slimming down the karaf core by eliminating
        that management overhead would be good for something like
        karaf-boot.  In the current monolithic enterprise environment it
        makes sense to have stacks of features available to load when
        necessary.  The static mechanism would get rid of that. But it
        would also permit building up a centralized registry or library
        of features that one could leverage by adding into a boot recipe
        of some sort.   Maybe we'd get the quick flexibility for
        creating projects that archetypes always seemed to promise but
        never quit seemed to manage.

        One item that has always bothered me about features is they are
        orthogonal to but replicate much of what goes into a POM.   One
        ends up with two sets of dependency management mechanisms that
        have never really dovetailed.  It would be nice if we had a
        Maven plugin that could look at the dependencies in a POM and
        create a static feature profile or at least give a good guess at
        what they should be while allowing for some tweaking.  Perhaps
        since the karaf-boot environment is static and doesn't rely on
        or expect another environment to provide dependencies that wouldkie-aries-blueprint
        be easier to accomplish.

        Funny how Moore's law took a sideways turn on us.  Now we don't
        have the ever increasing clock speed but we have cores coming
        out our ears and RAM and disk space in abundance.  A little
        fatness in our deployments is an acceptable trade off now.

        Brad


    --
    Christian Schneider
    http://www.liquid-reality.de

    Open Source Architect
    http://www.talend.com






--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration





-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com
Reply | Threaded
Open this post in threaded view
|

Re: MicroWebserver

Jean-Baptiste Onofré
In reply to this post by Brad Johnson
Right now, the features XML can already define a OBR repository.

I don't think the opposite makes sense and solve the problem. It's
something that we already discussed, but I'm not convince driving all
with OBR repo (the new name is Bundle Repository ;)) will be so good. It
will be a mess IMHO and we will have some missing parts compared to
features. I worked in Apache ACE (mostly focused on OBR repository and
deploy admin), and parts are missing when dealing with more than simple
bundles.

Regards
JB

On 04/18/2016 05:54 PM, Brad Johnson wrote:

> Christian,
>
> I think your idea of the OBR repository for features and profiles is
> good.  I think when I start using karaf-boot what I'd certainly want is
> a way to specify features almost like recipes.  Those would obviously
> have to be inlined in the bundles features file as their true
> mvn:group/artifact/version form since everything at that point is
> static.  We needn't have a look up mechanism at runtime obviously. In
> similar fashion one could have a plugin look at the POM and dependencies
> specified as being required for running the bundle and have those
> installed. Unfortunately I don't know how Karaf-boot is going to install
> those.  In lib? endorsed?  I'd think there would be a number of possible
> ways to add those and the static nature will simplify much of it.
>
> Brad
>
>
>
> On Mon, Apr 18, 2016 at 2:29 AM, Christian Schneider
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     I am not very familiar with the concept of profiles till now.
>     Guillaume explained it a bit to me but honestly I hope we do not
>     need it too often. An OSGi application should be mainly built out of
>     bundles and profiles concentrate more on the non OSGi static files
>     in karaf.
>
>     The main part of the packaging in karaf boot is to resolve features
>     at build time. It uses the static profile but nothing else from the
>     profiles concept. This part already works. I have created such a
>     packaging for the tasklist-ds example:
>     https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist-ds/app
>
>     The second part. The karaf boot development model is still in the
>     making but you can already use the packaging if you want.
>
>     About features and poms you are right. You have to maintain two sets
>     of dependencies but this is just a result of the fact that OSGi
>     resolves differently from maven. So this never will completely go
>     away. What we could do with a pom is create an obr repository. This
>     could then be used to back a feature repo. So currently in a feature
>     we have to list all the bundles, with a backing obr repo we could
>     instead only list some requirements (like top level bundles) and
>     resolve the rest.
>
>     I have prototyped such a packaging using bndtools as it already has
>     the OBR resolving:
>     https://github.com/cschneider/bndtools-tutorials/tree/master/tasklist-ds
>     The creation of the repository went quite smoothly as eclipse m2e
>     provides nice tooling to define the pom and the necessary excludes.
>     Defining the requirements in bndtools also works really nicely as
>     they got a good UI for it.
>
>     Unfortunately bndtools does not know about features. So while this
>     approach is better than defining each single bundle of an
>     application it is not really better than the karaf approach were you
>     define most of your deps as features. So this would only be really
>     useable once we have both features and a backing obr repo.
>
>     Another thing that I found bndtools is lacking is all the
>     preparation karaf already has for some of the more complicated
>     features like tansactions and cxf. There it is not enough to just
>     install bundles.
>     You also have to fiddle with the boot path for transactions and some
>     java se spec nightmares in the case of cxf. So these are the regions
>     where karaf really helps.
>
>     Christian
>
>     On 18.04.2016 07:24, Brad Johnson wrote:
>
>         Christian,
>
>         I just re-read your section on the static profiles.  That makes
>         sense.  I could see this becoming like puppet/chef sort of
>         recipes or even like the way Docker allows building up of
>         images.  Not that I know those that well.  I use features all
>         the time of course but see huge advantage to making that a
>         bundle time set of static steps instead of the current runtime
>         usage.  Not that the current use is bad, mind you, its context
>         is different and slimming down the karaf core by eliminating
>         that management overhead would be good for something like
>         karaf-boot.  In the current monolithic enterprise environment it
>         makes sense to have stacks of features available to load when
>         necessary.  The static mechanism would get rid of that. But it
>         would also permit building up a centralized registry or library
>         of features that one could leverage by adding into a boot recipe
>         of some sort.   Maybe we'd get the quick flexibility for
>         creating projects that archetypes always seemed to promise but
>         never quit seemed to manage.
>
>         One item that has always bothered me about features is they are
>         orthogonal to but replicate much of what goes into a POM.   One
>         ends up with two sets of dependency management mechanisms that
>         have never really dovetailed.  It would be nice if we had a
>         Maven plugin that could look at the dependencies in a POM and
>         create a static feature profile or at least give a good guess at
>         what they should be while allowing for some tweaking.  Perhaps
>         since the karaf-boot environment is static and doesn't rely on
>         or expect another environment to provide dependencies that would
>         be easier to accomplish.
>
>         Funny how Moore's law took a sideways turn on us.  Now we don't
>         have the ever increasing clock speed but we have cores coming
>         out our ears and RAM and disk space in abundance.  A little
>         fatness in our deployments is an acceptable trade off now.
>
>         Brad
>
>
>     --
>     Christian Schneider
>     http://www.liquid-reality.de
>
>     Open Source Architect
>     http://www.talend.com
>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com
12