Twitter github

Author Archive

KubeCon CloudNativeCon 2017 Takeaways + 2018 Predictions

It was a crazy 2017 for me with 300,000 miles of business travel, but it was all worth it to experience every major cloud provider adopt Kubernetes in some fashion and grow our community to 14 projects total! Also, it was amazing to help host 4000+ people in Austin for KubeCon/CloudNativeCon, where it actually snowed!

I’d like to share some personal take aways I had from the conference (of course with accompanying tweets) that will serve as predictions for 2018:

Exciting Times for Boring Container Infrastructure!

One of the themes from the conference was that the Kubernetes community was working hard to make infrastructure boring. In my humble opinion, Kubernetes becomes something like “POSIX of the cloud” or “Linux of the Cloud” where Kubernetes is solidifying kernel space but the excitement should be in user space.

The Open Container Initiative (OCI) community also held a meeting where it celebrated its v1.0 release to make containers a bit more boring and standardized.

In 2018, look for the boring infrastructure pattern to continue, the OCI community is planning to make distribution a bit more boring via a proposed distribution API specification. I also predict that some of the specialized/boutique cloud providers who haven’t offered Kubernetes integration will do so finally in 2018.

CNCF + KubeCon and CloudNativeCon: Home of Open Infrastructure

CNCF has a community of independently governed projects, as of today which there are 14 of covering all parts of cloud native. There’s Prometheus which integrates beautifully with Kubernetes but also brings modern monitoring practices to environments outside of cloud native land too! There’s Envoy which is a cloud native edge and proxy, that integrates with Kubernetes through projects like Contour or Istio, however, Envoy can be used in any environment where a reverse proxy is used. gRPC is a universal RPC framework that can help you build services that run on Kubernetes or any environment for that matter! There are many other CNCF projects that have use cases outside of just purely a cloud native environment and we will see more of that usage grow over time to help companies in transition to a cloud native world.

In 2018, look for CNCF conferences continue to grow, expand locations (hello China) and truly become the main event for open source infrastructure. In Austin it was incredible to have talks and people from the Ansible to Eclipse to JVM to Openstack to Zephyr communities (and more). I can’t think of any other event that brings together open source infrastructure across all layers of the cloud native landscape.

Moving up the Stack: 2018 is Year of Service Meshes

Service meshes are fairly a new concept (I highly recommend this blog post by Matt Klein if you’re new to the concept) and will become the middleware of the cloud native world. In CNCF, we currently host Envoy and linkerd which helped poineer this space. In 2018, I expect more service mesh related projects to be open sourced along with more participation from traditional networking vendors. We will also see some of the projects in this space to mature with real production usage.

Cloud Native AI + Machine Learning: Kubeflow

In 2018, ML focused workloads and projects will find ways to integrate with Kubernetes to help scale and encourage portability of infrastructure. Just take a look at the kubeflow project which aims to make ML with Kubernetes easy, portable and scalable. Note, this doesn’t mean that AI/ML folks will have to become Kubernetes experts, all this means is that Kubernetes will be powering more AI/ML workloads (and potentially even sharing their existing cloud native infrastructure). I expect more startups to form in this space (see RiseML as an example), look to see a “cloud native” AI movement that focuses on portability of workloads.

Developer Experience Focus and Cloud Native Tooling

One of my favorite keynotes from KubeCon was Brendan Burns speaking about metaparticle.io, a standard library for cloud native applications. I completely agree with his premise that we need to democratize distributed systems development. Not everyone developer needs to know about Kubernetes the same way not every developer needs to understand POSIX. In 2018, we are going to see an explosion of open source “cloud native languages” that will offer multiple approaches to democratizing distributed systems development.

Also in 2018, I expect us to see growth in cloud native development environments (IDEs) to provide better developer experience. As an example, for those that were wondering why there was an Eclipse Foundation booth at KubeCon, they were demoing a technology called Eclipse Che which is a cloud native IDE framework (your workspace is composed of docker/container images). Che is a framework that helps you build Cloud Native IDEs too, for example, OpenShift.io is OpenShift integrated with Che to provide you a fully blown online development experience.

Finally in 2018, I expect the developer experience of installing Kubernetes applications improved, including the underlying technology for doing so. For example, the Service Catalog work and websites like kubeapps.com showcase what is possible in making it easier for people to install Kubernetes app/integrations, we’ll see this grow significantly in 2018. Also I predict that the Helm community will grow faster than it has before.

Diversity and Inclusion

One of my favorite take aways from the conference was the focus on diversity and inclusion within our community:

We (thank you amazing diversity committee) raised $250,000 and helped over 100 diversity scholarship recipients attend KubeCon/CloudNativeCon in Austin. In 2018, I predict and truly hope some other event will match or beat this.

Anyways, after a crazy 2017, I can’t wait to grow our communities in 2018.

 

 

Cloud Native Computing Foundation 2 Years Later

A little over two years ago after five years of service at Twitter, I took the opportunity to build an open source foundation from scratch using some of the computing techniques we experimented with at Twitter:

https://twitter.com/cra/status/937460286210048000

I was initially excited about the idea because of my experience with open source foundations in previous lives, from being involved with the Eclipse Foundation, Linux Foundation, Apache Foundation plus part of the early discussions around OpenStack governance formation. I viewed this as an opportunity to learn from the lessons of other foundations and do something new and modern in the GitHub era, along with of course making our own mistakes. You really don’t get many opportunities to start an open source foundation from scratch that will impact the whole industry.

Stepping back, the original idea behind the Cloud Native Computing Foundation (CNCF) was to promote a method of computing (we call it cloud native) pioneered by internet scale giants such as Google, Twitter, Facebook and so on and bring it to the rest of the industry. If you looked inside these companies, you can see they were running services at scale, packaged in containers, orchestrated by some central system.

The first mission was to provide a neutral home for Kubernetes as the seed project of the foundation but also provide room for adjacent projects that specialized in specific areas for this new world (think monitoring and tracing as an example). The second mission was to convince all the major cloud providers to build in Kubernetes as a managed offering so we could essentially have a “POSIX of the cloud” that would give us a set of distributed APIs that would work everywhere (including on premise). Last week with AWS announcing their managed offering EKS, we have accomplished this goal with every major cloud provider supporting Kubernetes natively, kubernetes is truly the lingua franca of the cloud.

We still have a long way to go within CNCF to truly making cloud native computing ubiquitous across the industry, but I’m excited to see so many companies and individuals come together under CNCF to make this happen, especially as we have our largest annual gathering this week, KubeCon/CloudNativeCon. Personally, I’m nothing but thrilled to what the future holds and truly lucky to be serving our community under the auspices of the foundation.

A special thank you to Craig McLuckie, Sarah Novotony, Todd Moore, Ken Owens, Peixin Hou, Doug Davis, Jeffrey Borek, Jonathan Donaldson, Carl Trieloff, Chris Wright and many other folks that were at that CNCF first board meeting two years ago bootstrapping the foundation.

Top 6 Public Cloud Providers in CNCF

Last week, I had the pleasure of welcoming Oracle into the Cloud Native Computing Foundation (CNCF). This now marks the top 6 leading public cloud providers in the world are now part of the CNCF:

This also marks for the first time in the history of our industry that these leading cloud providers are working together in the same open source focused foundation to move the state of the art infrastructure forward.

Also last week I had the opportunity to bring in two new high quality cloud native projects into CNCFEnvoy is a high-performance open source edge and service proxy that makes the network transparent to applications. Jaeger is an open source distributed tracing system inspired by Google Dapper paper and OpenZipkin community. It can be used for tracing microservice-based architectures. Uber began deploying Jaeger internally in 2015. It is now integrated into thousands of microservices and recording thousands of traces every second.

 

Anyways, this is one of the reasons I enjoy working in open source today, bringing together diverse (and even competing) companies to build a better world by collaborating in the open!

Launching TODO Group Guides

In my participation with the TODO Group, we recently published a set of open source guides (under CC-BY-SA 4.0) dedicated to help companies build out open source programs:

In the last year or so, we have seen companies like AWS build out an open source program via @AWSOpen and even companies like VMWare hired their first Chief Open Source Officer.

We’ve had many organizations approach TODO Group members asking for advice on how to get started with an open source program and these guides are a reaction to that. It is our hope that these are living guides and evolve with community input over time.

Shipping OCI v1.0.0

Last month after nearly 2 years, the Open Container Initiative (OCI) community shipped v1.0.0 (see release notes) of two container standard specifications:

Outside of relevant container standards finalized, another thing that was enabled by this v1.0.0 release was an interesting IP Policy (OWFa v1.0) embedded in the OCI charter. Essentially, there is patent non-assertion and royalty free patent grant made by all OCI members against the entire 1.0.0 specification implementations, not just their contributions!

https://twitter.com/cra/status/899668238824591361

This is simply good news for the industry when it comes to what I call establishing an “IP no fly zone” for core container technology, which should spur further container adoption. Also, standardization also helps implementors and tool builders feel safe that things won’t break, which has been a bit of a challenge given how fast container technology has been moving.

Finally, one more thing to highlight is that OCI is also helping sponsor a diversity scholarship at DockerCon EU in Copenhagen:

Please encourage qualified folks to apply, the deadline is September 5th.

AWS Joining CNCF

It’s been a little over a year and a half since I started to help build the Cloud Native Computing Foundation (CNCF) from scratch. One of our original goals was to build a modern open source foundation focused on a new form of cloud computing called “cloud native” (essentially think of microservices that run in containers that are orchestrated) and to get all the major cloud providers at the table to adopt this form of computing.

Last week, we were happy to welcome AWS to CNCF as a member and that now brings us to having the top five cloud providers in the world at the table committing to adopting and promoting cloud native computing:

First off, it’s always great to see an original vision of when we started CNCF come into reality, you can read more from Adrian Cockcroft why they decided to join CNCF and support cloud native computing.

Second, I think it’s great to see a company like Amazon expanding its open source efforts as they where one of the last large companies without a formal open source program. They recently started an official open source program office @AWSOpen under the leadership of Adrian Cockroft and Zaheda Borat and it’s been great to have them participate in the TODO Group too!

Anyways, always great to see large and impactful companies increase their commitment to open source. Now it’s interesting to think what large companies out there don’t have an official open source program or strategy (I’ll leave this as an exercise to the reader).

Become a Founding Kubernetes Certified Service Provider

In early September, CNCF will be announcing the founding class of Kubernetes Certified Service Providers (KCSPs). If your company provides professional services to support Kubernetes deployments, please consider signing up to become part of the founding class.

The main benefits of becoming a KCSP are:

  • Placement in a new section at the top of https://kubernetes.io/partners/
  • Monthly meetings with cloud native project leaders, TOC members,
    and representatives from the CNCF Governing Board
  • Access to leads from end users looking for support

Requirements are:

  • Three or more engineers who pass the Certified Kubernetes Administrator (CKA) exam
  • Demonstrable activity in the Kubernetes community including active contribution
  • A business model to support enterprise end users, including putting engineers at a customer site

The CKA exam is about to enter early release beta testing prior to the public release in September. It is an online, proctored, performance-based test that requires solving multiple issues from a command line. It takes 3 to 4 hours to complete, and costs $300, though a discount is available for beta testers to $100.

If your company is interested in becoming a KCSP, please do the following 4 things:

  1. Ensure that your company is listed at https://kubernetes.io/partners/
    and if not (or if the listing should be updated), please do so via the link
    at the top of that page.
  2. Have 3 or more of your Kubernetes experts sign up for the beta test at:
    https://docs.google.com/forms/d/e/1FAIpQLSd9-6nL5L3SzWIddCSPoKeuX_Pdq_KHI8C4mQzcUryP-gu0dQ/viewform
    .Please have them use their company email so we can properly associate
    them. Within a week, we will send beta test dates, a discount coupon code, and instructions to register and schedule.
  3. Register your interest in becoming a KCSP at this form: https://docs.google.com/forms/d/e/1FAIpQLSfai-zlNuvP-q0fz3jw89v3v4m_wYaF7tOBmNY0WoKsZgeQUQ/viewform
  4. If you are not already on it, and want to track progress of the certification program over time, please subscribe to the Kubernetes Certification Working Group list: https://lists.cncf.io/mailman/listinfo/cncf-kubernetescertwg.

Questions or issues? Please email cncf-kcsp-support at lists.cncf.io

Thanks!

2018 Eclipse Simultaneous Release Name: Photon

Hey Eclipse friends, as part of my duties on the Eclipse Planning Council, we have worked with the Eclipse Foundation EMO and wider community to finalize the Oxygen+1 simultaneous release name (see bug 510201 for details): Photon

Thanks to everyone who voted in the community!

Open Container Initiative in 2017

Last year I had the opportunity to help build out/run the Open Container Initiative (OCI) and I wanted to take some time to reflect back on what the OCI community accomplished in 2016 and how far we’ve come in a short time since we were founded a little over a year ago.

The community has been busy! The project saw 3000+ commits from 128 different authors across 36 different organizations. With the addition of the Image Format specification project, we expanded our initial scope from just the runtime specification. Our membership grew to nearly 50 members and we also added new developer tools projects —runtime-tools and image-tools— which serve as repositories for conformance testing tools and have been instrumental in gearing up for the upcoming v1.0 release.

We’ve also recently created a new project within OCI called go-digest (which was donated and migrated from docker/go-digest). This provides a strong hash-identity implementation in Go and services as a common digest package to be used across the container ecosystem.

In terms of early adoption, we have seen Docker support the OCI technology in its container runtime (libcontainer) and contribute it to the OCI project (as runc). Additionally, Docker has committed to adopting OCI technology in its latest containerd announcement. The Cloud Foundry community has been an early consumer of OCI by embedding runc via Garden as the cornerstone of its container runtime technology. The Kubernetes project is incubating a new Container Runtime Interface (CRI) that adopts OCI components via implementations like CRI-O and rklet. The rkt community is adopting OCI technology already and is planning to leverage the reference OCI container runtime runc in 2017. The Apache Mesos community is currently building out support for the OCI image specification.

Speaking of the v1.0 release, we are getting close to launch! The milestone release of the OCI Runtime and Image Format Specifications version 1.0 will hopefully be available this first quarter of 2017 or shortly the following quarter, drawing the industry that much closer to standardization and true portability. To that end, we’ll be launching an official OCI Certification program once the v1.0 release is out. With OCI certification, folks can be confident that their OCI-certified solutions meet a high set of criteria that deliver agile, interoperable solutions.

We’ll be looking into the possibility of adding more projects in the coming year, and we hope to showcase even more demonstrations of the specs in action under different scenarios. We’ll be onsite at several industry events, so please be on the lookout and check out events page for details.

There is still much work to be done!  The success of our community depends on a wide array of contributions from all across the industry; the door is always open, so please come join us in shaping the future of container technology! In particular, if you’re interested in contributing to the technology, we recommend joining the OCI developer community which is open to everyone. If you’re building products on OCI technology, we recommend joining as a member and participating in the upcoming certification program.

Note: This was cross-posted to the OCI community blog.

Improved GitHub Code Review Tools

The default GitHub code review experience has always been lacking for me, especially when it comes to code review enforcement. I definitely admit to my bias of being a Gerrit code review fan but at least GitHub is moving in the right direction with recent code review improvements earlier this year.

On the bright side, GitHub has a fairly open API and there’s been a great ecosystem of tools that have popped up that help deal with some of the shortcomings in my opinion. Even more so after GitHub opened up the commit status API, it’s definitely enabled some interesting workflows.

PullApprove

One interesting code review workflow involves enforcing that changes must be reviewed by certain team members (which is a fairly common code review practice). In my opinion, the best tool that I’ve come across that handle this scenario is PullApprove. All you need is a simple metadata file that formalizes your requirements, and you’re good to go, here’s an example of what we use in the OCI project:

approve_by_comment: true
approve_regex: '^(Approved|lgtm|LGTM|:shipit:|:star:|:\+1:|:ship:)'
reject_regex: ^Rejected
reset_on_push: true
signed_off_by:
required: true
reviewers:
teams:
- image-spec-maintainers
name: default
required: 2

The important pieces here are the approve/reject regexes which dictate what terms will be used to approve and reject changes:

PullApprove

As a bonus, the “signed_off_by” setting can be used to enforce DCO which is to make sure your commits are properly signed off for DCO purposes.

LGTM

I also want to give an honorable mention to LGTM which is a simple pull request approval system that is open source but not as configurable as PullApprove IMHO.

Reviewable.io

For any large scale and high velocity projects on GitHub, managing what you need to review can be a daunting task. I’ve personally found Reviewable.io as an interesting tool that helps address the problem of managing the context of what you looked last and what’s on your queue.

Screen Shot 2016-08-04 at 9.49.55 AM

Anyways, hope this helps and happy code reviewing! If you’re interested in more integrated GitHub code review tools, I highly recommend checking out the GitHub integration directory.