What is cloud broker??
First I will start by explaining what is cloud brokerage, a cloud broker is an intermediary provide one interface to manage all the complexity behind dealing with different cloud services.
If you are an IT enterprise that uses one cloud (public or private) you don’t need that brokering services, all you have to do is to understand that cloud API and start consuming them. But when organisations start using the cloud seriously the one cloud scenario will be likely the most common scenario. Because cloud offers different services for different use case, it provide infrastructure services, storage services, software as a service, data subscription and so long, having one cloud providing all services perfectly is a hard thing to have, for example let’s consider a software company that uses openstack cloud as private cloud to host intern management applications where security really matters, uses Amazon cloud as test platform to do some different deployment scenarios on different environments and finally they uses windows azure as PAAS. Let’s take a look to what this company must deal with:
-Mastering all the APIs of different clouds (OpenStack, Amazon and Azure)
-Having different monitoring system for deferent clouds
-Managing the security policies for each cloud
-Managing the billing system for each cloud (for example Amazon and azure have different billing policy)
As you see this company have many challenges to face and must have different teams for different cloud what make using those services very costly and very hard witch make no sense at all because cloud computing supposed to offer simplicity and chip services. So the best solution for this company is to use Cloud Service Brokerage(CSB) that offers one entry point to manage all the cloud providing one API, one security system, one monitoring system…(for more information read The role of a Cloud broker
http://www.cio.com.au/article/399699/role_cloud_broker/) so in the future CSB will have a very important role in the world of cloud computing and of course will have an economical influence where company will pay for brokering services. In order to help IT managers, CEO and any decision makers I will present the first open source CSB created by French research group, the project called Compatible one.
What is Compatible one??
Compatible one is the first open source cloud broker, it’s a collaborative project, the first release was on October 2011 and the final one will be on December 2012, the project members include industry and academics leaders, innovative technology statups and a world-renowned open source community. For more information visit the web site
http://compatibleone.org.
Compatible one component
The compatible one is composed of four parts:
Designer: Taking into account the user requirements
Scheduler: Design and construct of the provisioning plan
Broker: Execution of the provisioning plan
Provider: Deliver the request cloud service
Designer:
The designer will take user requirements in term of both technical and economic criteria, for example the user want to create an instance that have 3 cpu 16 G of memory and 30 G of hard drive and he want to pay no more than 1$ per hour. The user will put all that in a manifest file (well-structured XML file), the manifest structure is based on the model described by CORDS (CompatibleOne Resource Description Schema). CORDS is a model defined by the CompatibleOne Developers based on OCCI (Open Cloud Computing Interface).
What is OCCI?
OCCI is a community effort in order to standardize the cloud computing interface, cloud computing standardization is in it’s very early stages and OCCI is an effort that aims in its early start to provide one single interface to manage IAAS, and now it serve many other models like PAAS and SAAS
http://occi-wg.org/.
But before continuing with CORDS description I would like to explain how OCCI works:
OCCI is an intermediary between the client and the provider. The idea is to unify the way how client consume services by providing standard REST APIs called OCCI, OCCI interface will receive the Rest request from the client and then it will mapped to the resource management framework(RMF) of the cloud provider and finally the RMF will create some management resources according the user request and send back information to the RMF, the RMF mapped to the OCCI interface and finally OCCI will send a HTTP response to the user communicating all the information that he need to start using those managed resources.
OCCi Components:
OCCI define the infrastructure model extension witch is an abstraction model to real life resources it defines a Resources class composed of three classes:
Network: the user instantiate this class to define his Network resource needs (vlan, label (Token), state)
Storage: the user instantiate this class to define his storage needs (size, state)
Compute: the user instantiate this class to define his Network needs (Architecture, cores, memory…)
And finally a Link Class, the link class is responsible for associating one resource with another.
Also OCCI defines Mixin, the Mixins are extensions mechanisms that allow new resources capability to be added dynamically to a collection of resources instances (Mixin can be added to resource instance at creation time or at run-time but Mixin can be added only to instances and not objects).
The only question is how to control Mixin since is all dynamic, that’s mean how the user will now what’s supported and when he must add mixins. Well according to the OCCI the Core model is the solution for all those questions, this a fundamental feature that give the capability to resources to be added and discovered at run-time, OCCI users can connect to OCCI core model without having any idea about available resources and start discovering at run-time the various resources and links supported by that implementation.
But how??
If we examine the figure bellow will find that the structure of the class diagram become more complex, now Resource and Link both inherit from an abstract class called Entity, each Entity is identified by a unique Kind instance. The Kind type is actually a specialised type of category, Kind add to category Action instances, the Action instance represent an invocable operation applicable to a resource instance (e.g. for Virtual Machine resource Action can be launch). For compliance with OCCI Core, all of the types defined in the OCCI Core Model MUST be implemented.
The core Model assures the identification and classification of Entities (Resources and Links) witch make them discoverable. Because each Entity is associated to a unique identifier(Kind instance) so whatever a resource is added the OCCI will look for new Kind instances and find Entities instances associated to them witch make all the inheritance structure of Entity, Resource and Link is client discoverable. The Kind type must implement all category attributes and all instances of Kind type must follow those rules:
A unique Kind instance Must be assigned to each and every sub type of Entity, including Entity itself.
A Kind instance Must expose the attribute names of the Entity sub-type it identifies.
A Kind instance Must expose the Actions defined for its sub-type.
A Kind instance Must be related directly or indirectly to the Kind instance of Entity
If type B inherits type A, where A is a sub-type of Entity, the Kind instance of B MUST be directly related to the Kind instance of A.
The Figure illustrates the relationship of the Kind instances assigned to the Entity, Resource and Compute types. Compute inherits Resource and therefore the Kind instance assigned to compute is related to the Kind instance of Resource. The same applies to the Resource type which inherits Entity.
Example of simple VM resource creation
We can see that the requested Resources is compute and we added two mixin one associated to the OS(Ubuntu) of the virtual Machine and another one indicate the capacity requested(small machine). So the provider did the job and gives back a HTTP response indicating the URL associated to this managed resource.
So this is a brief explanation of OCCI specifications that help you understand better the CORDS schema.
Now let’s go back and examine CORDS specifications, CORDS it just an extension of the OCCI specification it adds many new objects like Monitoring, deferent network configurations, some node constraints… but in the same time they keep the compatibility with OCCI by implementing the OCCI core model.
The figure present a logical view of the CORDS, we can see that the implementation of OCCI core model is respected while adding many new objects.
The figure present CORDS virtual instance view. The user can define a contract where he put the profile of the service that he like to get and the list of providers, each service is composed by 1 or many contracts and each plan is composed by one or many services (All the attributes if CORDS are defined in the document CORDS reference Manual http://compatibleone.org/bin/download/Docs/WebHome/CordsReferenceManualV2.03.pdf). Finally after doing all the definition a Manifest must be defined as output to the Design phase.
Manifest version 2 description
Image components: each image has a system and packages to be added example (Ubuntu system + Mysql package)
Infrastructure components: Storage resources(example open stack volumes), Network and computes(small instances, tiny instances…)
CompatibleOne scheduler :
The User manifest is received by a mechanism called the SLAP Master allocation engine. This engine triggers the gathering of events required for accounting and billing, and then transmits the manifest to the CORDS Parser. The CORDS Parser transforms the XML description of the manifest file into a validated and well-formed provisioning plan. The CORDS Parser performs a syntactic verification of the manifest file prior to engaging the descriptive resources used to represent the resulting architectural plan required to meet the specific requirements (scheduler definition from the web site)
To do the parsing on the Manifest file a command line provided by the compatible one APIs can be used: co-parser manifest, that command line will produce a provisioning plan.
ComaptibleOne broker
The provisioning plan is transmitted to the CORDS Broker. The CORDS Broker is in charge of executing the provisioning plan for the production of an actual service instance to be delivered to the user. The broker will instantiate a service graph for the control of the contractual components described in the plan. During this operation the CORDS Broker will collaborate with various CompatibleOne service providers as required for the realization of the plan. This will be performed while respecting the different constraints and conditions expressly defined in the plan, such as an carrier selection, optimization, monitoring or accountancy.(broker definition from the web site)
To start the broker: $ broker --config accords.xml BROKER/1.0
To publish a provising plan : $ testcb --publisher http://127.0.0.1:8086 plan_manifest.xml
CompatibleOne services used by the broker:
COES:This component will examine monitoring information in the forms of provisioning alerts which will be used in the elaboration of the scoring information that will be used to influence the selection of particular provider platforms during subsequent placement operations.
CONETS: The CompatibleOne Network Services
COIPS: CompatibleOne Image Production services.
COSACS: CompatibleOne Software Appliance Services this components used to personalize application images during resource deployment startup(example adding requested packages)
PS: All the services used by the broker are standalone OCCI (OCCI implementation) REST servers that must be run before start using the broker.
Broker will produce Contracts for the providers
CompatibleOne Operator
The CORDS Broker coordinates the activity of the CORDS Procci for the selection of the specific cloud providers for the delivery of the provisioning services required by the consumer. A particular provider's ability to deliver resources is guaranteed by their declaration of presence, and their consequently available resources, to the CORDS Publisher. Providers are selected by the CORDS Procci as directed either by the constraints described in the provisioning plan or specified in the global configuration of an operational instance of the ACCORDS platform. ACCORDS (Advanced Capabilities for CompatibleOne Resources Distribution Services) is actually the Provisioning System for CompatibleOne
Communication with the Provider is performed through standard CORDS OCCI client/server interfaces between the CORDS Procci, on the CompatibleOne side, and the Provider Procci, on the Provider side. In this way a individual and specific Procci service provider is responsible for interfacing with a particular Cloud Provider, e.g. an OpenNebula Procci, an OpenStack Procci, an Amazon Procci or an Azure Procci.
Consequently, in order to be made available for operation and use within ACCORDS, a resource needs only to implement an OCCI client/server Procci interface.
Publisher: the publisher is responsible for publishing service provider collection of categories (OCCI specification) for use by other components.
The publisher is a standlone OCCI REST server and must be started before any of the other components may be used. The publisher is started by the command:
$ publisher --config accords.xml PUBLISHER/1.0
Resolver: The resolver library allows the publisher to be consulted for discovery of the collection of service providers for a particular category of service.
Resolver are C libraries, any category can be inspected by typing this command line
$ testresolver categoryname1 categoryname2
Procci: This component of the ACCORDS platform is responsible for the management of generic provisioning contracts as requested by the service broker during the creation of the service instance control graph. Provider specific contracts will be negotiated here using the provider specific procci components as required by the description of the node to be provisioned.
