Wilson Mar bio photo

Wilson Mar

Hello!

Calendar YouTube Github

LinkedIn

GitOps

ArgoCD (for Kuberetes Helm), Flux from WeaveWorks, etc.

US (English)   Norsk (Norwegian)   Español (Spanish)   Français (French)   Deutsch (German)   Italiano   Português   Estonian   اَلْعَرَبِيَّةُ (Egypt Arabic)   Napali   中文 (简体) Chinese (Simplified)   日本語 Japanese   한국어 Korean

Overview

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

GitOps

“GitOps” is a set of best-practices for deployments. Its strategy is use of a text file Source Version Control system (SVC such as GitHub or GitLab) to create resources based on Infrastructure as Code (IaC), then run tests using what it created. Relying on versioned code makes for easier rollbacks, straightforward auditing, better traceability.

Some “old timer cowboys” may bristle at their “loss of control” because when manifests are version controled are considered the “one source of truth” for defining the desired application state, there cannot be ad hoc changes “on the fly” using CLI, which leaves no trace of who made what changes when, thus causing configuration drift. A deployed application whose live state deviates from the target state is considered “OutOfSync”.

Pros and cons:

  • https://humanitec.com/blog/gitops-pros-and-cons
  • https://blog.container-solutions.com/gitops-limitations

GitOps Vendors

References:

  • VIDEO: “Complete hands-on workshop”

Both Flux and ArgoCD are CI on dev code & CD to prod:

  • incubated as a sandbox project under CNCF for Kubernetes for OpenGitOps
  • serve the same purpose
  • supports Webhood Receivers for on-demand reconciliation for faster commits
  • supports RBAC (Role-Base Access Control)
  • supports multi-tenancy
  • supports Helm and Kustomize for customizing application configuration in a template-free way, a core K8s tool shipping with kubectl.
  • supports secrets management
  • Does Garbage collection (Pruning)

But each take a different approach:

  • https://github.com/open-gitops/documents/pull/47
  • https://thenewstack.io/gitops-on-kubernetes-deciding-between-argo-cd-and-flux/
  • https://opengitops.dev/
  • https://opengitops.dev/blog
  • https://github.com/open-gitops/documents
  • https://slack.cncf.io/ <- join #opengitops and #wg-gitops channels.
  • https://rajputvaibhav.medium.com/argo-cd-vs-flux-cd-right-gitops-tool-for-your-kubernetes-cluster-c71cff489d26
  • https://www.reddit.com/r/kubernetes/comments/p8smmi/flux_vs_argo/
  • https://earthly.dev/blog/flux-vs-argo-cd/
  • https://zelarsoft.com/gitops-on-kubernetes-deciding-between-argo-cd-and-flux/
  • https://blog.container-solutions.com/fluxcd-argocd-jenkins-x-gitops-tools
  • https://www.sgmoratilla.com/2021-10-28-flux-vs-argocd/
  • https://www.doit.com/avoid-fluxcd-for-gitops/

Flux2 ArgoCD
CLI-first, so simpler for ops.
(Web UI plugin) 		Both CLI and
WebUI for self-service by devs
Bootstraps to a GitHub repo Install via Helm YAML manifests
Variable substitution -
Common interface (No clusters) Cluster Hub connects to many K8s clusters w/out an Argo cluster
- Manual commit and sync to update containers
CLI-first (Web UI plugin), so simpler Both CLI and WebUI
- No Helm CLI
App delivery using Kustomize Controller referencing dependsOn dependencies Can also apply raw YAML directly as syncwaves and phases
New app in new repo New app in same project repo
Helm Controller uses native Helm Golang library Overtakes Helm to render template files and pipes to kubectl
Relies on K8s RBAC-bound roles (no Flux users) Separate ACL RBAC (OIDC), also SSO
- also supports ksonnet & jsonnet to secure manifests
Automated container updates Manual commit and sync to update containers
Reconciliation using GitOps Toolkit Global settings checked every 3 minutes by default
Monitors container repos ?

Argo CD, workflows, events, and rollouts (the big ones) can all be installed and used without using any of the others.

https://github.com/flux-subsystem-argo/flamingo

WeaveWorks Flux

Her companion blog has this flowchart of how the various Flux controllers work together with the K8s api-server: gitops-flux-955x792.jpg

  • https://github.com/fluxcd/source-controller Source controller
  • Kustomize controller
  • Helm controller

ArgoCD

  • https://argoproj.github.io/cd
  • https://argo-cd.readthedocs.io/en/stable/
  • https://github.com/aws-samples/eks-blueprints-add-ons
  • https://www.youtube.com/watch?v=MeU5_k9ssrs by Nana
  • https://github.com/clowdhaus/gitops-manifests
  • https://akuity.io/ is the enterprise offering

Argo CD is a declarative, “GitOps” continuous delivery tool for Kubernetes.

“Continous” because an ArgoCD GitOps agent controller is deployed in the Kubernetes cluster. Since ArgoCD is a Kubernetes resource, it can perform the equivalent of a kubectl apply command such as:

    kubectl apply -f my-application.yml

Several apps can be initiated by a multi-app yaml file.

NOTE that ArgoCD applications must be deployed in the same namespace as ArgoCD itself.

Real Time Container Registry

Since ArgoCD performs live updates, ArgoCD doesn’t reference static Container Registry images? This seems more secure because there are no static Container files sitting around to be corrupted if developer credentials are compromised, then used to maliciously alter assets.

ArgoCD processes Helm charts instead of helm commands. So helm commands don’t work because Argo CD took its place.

When a Git commit of a Kubernetes manifest, Helm chart, or Kustomize definition is detected, the GitOps agent gets Kubernetes to reach the same state as what is described in Git. That’s the declarative part.

  • Manifest changes can be detected by the ArgoCD agent periodically polling one or more Git repositories that contain Helm charts (or Kustomize files) which define applications. That can take up to 3 minutes (or whatever time you have set as sync period) for Argo CD to detect the changes.

  • Additionally, Git Webhooks can initiate the sync process as soon as there is any change in Git.

PROTIP: Argo CD can use a “pull” or the “push” approach of traditional CI/CD.

CodeFresh account

ArgoCD is created by developers within the CodeFresh corporation. So get a CodeFresh account:

  1. In a browser go to

    https://codefresh.io/codefresh-signup/

  2. Select a federated IdP (GitHub, Google, Azure, etc.).

  3. Confirm in your email client.

    A) Install ArgoCD for dev

    ArgoCD is open-sourced at https://github.com/argoproj/argo-cd/

    B) Install ArgoCD for production

    ArgoCD is one member of a family of projects from CodeFresh:

    • Autopilot

    • Argo Rollouts minimize downtime converting the Kubernetes Deployment to a Rollout resource to enable ArgoCD’s “Progressive Delivery Kubernetes ingress controller” CRDs for blue/green, canary, a/b, or fine-grained weighted traffic shifting based on feature flags. ArgoCD provides automatic rollbacks while Kubernetes does not. [tutorial]

    • Argo Workflows (Workflow engine for Kubernetes)

    • Argo Events (Event handling for Kubernetes)

    For Production use:

  4. Install ArgoCD onto a Kubernetes cluster by following this hands-on Instruqt lab.

  5. Link additional clusters to a single Argo CD instance. ArgoCD can deploy applications on the cluster it is installed on, or other external clusters that are authenticated correctly You can have any combination of clusters and ArgoCD instances.

  6. ArgoCD can manage its own upgrades if Autopilot is installed using:

    https://github.com/argoproj-labs/argocd-autopilot

Configure

IdP

  1. Use an SSO provider (such as Okta) for authentication.

    TODO: Setup

    Keeping Secrets secret

    https://argo-cd.readthedocs.io/en/stable/operator-manual/secret-management/

    If Bitnami is used, encrypted values are stored encrypted in a Git repo. Encryption of secrets happen via the kubeseal executable. Decryption happens via the Sealed Secrets controller.

    Install ArgoCD

  2. To create an ArgoCD namespace:

    kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

    Manifest Templates

  3. ArgoCD makes use of Helm charts native to Kubernetes:

    https://github.com/argoproj/argo-helm/tree/main/charts/argo-cd/templates

    https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/declarative

    Health Checks

    In addition to Kubernetes’ own health checks, Argo CD also keeps track of each deployed application’s service health:

    • “Healthy” -> Resource is 100% healthy (observed generation is equal to desired generation)
    • “Progressing” -> Resource is not healthy but still has a chance to reach healthy state
    • “Suspended” -> Resource is suspended or paused. The typical example is a cron job
    • “Missing” -> Resource is not present in the cluster
    • “Degraded” -> Resource status indicates failure or resource could not reach healthy state in time
    • “Unknown” -> Health assessment failed and actual health status is unknown

    Lua scripts can be written to define health checks of custom Kubernetes resources:

    • Argo Rollout (and associated Analysis and Experiments)
    • Bitnami Sealed secrets
    • Cert Manager
    • Elastic Search
    • Jaeger
    • Kafka
    • CrossPlane provider

    Health status is shown in the UI and or retrieved by CLI sync commands.

    Sync from GUI

  4. To synchronize the application because it hasn’t been deployed yet:

    argocd app sync "${ARGOCD_APP}"

  5. To confirm whether apps in a namespace is running:

    kubectl -n "${NAMESPACE}" get all

    The application will have a status “Running” if synchronized successfully.

    Sync and Self-Healing

    https://learning.codefresh.io/path-player?courseid=gitops-with-argo&unit=gitops-with-argo_6177d85a23d5bUnit

    An ArgoCD application is essentially a combination of a git repository, an Argo project, several sync options and other values.

  6. View a sample ArgoCD app:

    https://github.com/codefresh-contrib/gitops-certification-examples/blob/main/declarative/single-app/my-application.yml

    https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/

  7. Since someone can still perform Kubernetes kubectl commands, ArgoCD can reconcile (auto-sync) applications on its own. Enabling it includes specification by editing the argocd-cm configmap on the same namespace as ArgoCD:

    apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
  labels:
 app.kubernetes.io/name: argocd-cm
 app.kubernetes.io/part-of: argocd
data:
  timeout.reconciliation: 180s

    The timeout.reconciliation default is 3 minutes (60*3=180).

    =0s disables sync functionality.

  8. If “auto-sync” is enabled and an Argo CD application is deleted a resource in Git, Argo CD will automatically delete that resource from the cluster only if the auto-prune option is also set to “true”.

    Restart

  9. After a manifest is changed, redeploy the argocd-repo-server service.

    This Instruqt hands-on lab shows how to deploy and detect changes.

Videos

https://opengitops.dev/events lists videos at GitOpsCon.

Get involved

https://discord.com/channels/1001951070056099871/1001951070710415421

https://opengitops.dev/get-involved

https://argo-cd.readthedocs.io/en/stable/

https://learning.codefresh.io/start

  1. GitOps Fundamentals - Learn the basics of GitOps with ArgoCD and Argo Rollouts.
  2. GitOps at Scale - Learn sound strategies for managing applications and version deployments across
  3. GitOps at Edge - Learn how to deploy GitOps in your enterprise with confidence and minimal downtime.

References

https://learning.codefresh.io/course/gitops-with-argo

https://dzone.com/articles/what-is-gitops-1

GitOps was published on .