How To Setup NetApp Harvest Using Kubernetes (Using Pre-Built Container Image)
*2018-03-18: Updated for version 1.4.1 of my container image which includes bug fixes, Grafana 4.5.2 and NetApp Harvest 1.4
This post is based on the original “How To Setup NetApp Harvest Using Docker” blog post however it has been tweaked to reflect the use of Kubernetes and the NetApp Trident plugin. It is assumed both Kubernetes and NetApp Trident are already deployed so if you have questions on deploying these technologies see here.
-
- Image download and distribution
- Create the appropriate directories which will house the docker container build files
1# mkdir -pv /root/docker/harvest
- Download the custom NetApp Harvest Docker image (dburkland/harvest)
- Download the following files to your workstation and upload them to the “/root/docker/harvest/“ directory on the Kubernetes master node
- dburkland_harvest_v1.4.1aa
- dburkland_harvest_v1.4.1ab
- dburkland_harvest_v1.4.1ac
- dburkland_harvest_v1.4.1ad
- dburkland_harvest_v1.4.1ae
- dburkland_harvest_v1.4.1af
- dburkland_harvest_v1.4.1ag
- dburkland_harvest_v1.4.1ah
- dburkland_harvest_v1.4.1ai
- dburkland_harvest_v1.4.1aj
- dburkland_harvest_v1.4.1ak
- dburkland_harvest_v1.4.1al
- Uncompress the multi-part zip file
12345cd /root/docker/harvestcat dburkland_harvest_v1.4.1a* > dburkland_harvest_v1.4.1.zipunzip dburkland_harvest_v1.4.1.zipNOTE: At this point enter NTAPHarvestDocker2016 for the password in order to properly unzip the aforementioned zip file. - Import the Docker image file
123docker load < dburkland_harvest_v1.4.1.tarNOTE: You MUST perform this step on EVERY node in the Kubernetes cluster. It is recommended to deploy a local Docker image repository. For instructions on how to deploy one, see the "Sources" section below. - Verify you can now see the “dburkland/harvest” image in the Docker image inventory
1docker images
- Download the following files to your workstation and upload them to the “/root/docker/harvest/“ directory on the Kubernetes master node
- Create the appropriate directories which will house the docker container build files
- Deploy NetApp Harvest in the Kubernetes Environment
- Login to your kubernetes master node and create the harvest namespace
1# kubectl create namespace harvest - Define the persistent volume claims (PVCs) on your kubernetes master node
1234567891011121314151617181920212223242526272829303132333435# cd trident-installer# vi setup/pvc-dburkland_harvest_carbon.yamlkind: PersistentVolumeClaimapiVersion: v1metadata:name: var-lib-carbonnamespace: harvestannotations:volume.beta.kubernetes.io/storage-class: default-dburklandtrident.netapp.io/exportPolicy: "prod_kubernetes_nodes"trident.netapp.io/snapshotPolicy: "default-dburkland"spec:accessModes:- ReadWriteManyresources:requests:storage: 5Gi# vi setup/pvc-dburkland_harvest_conf.yamlkind: PersistentVolumeClaimapiVersion: v1metadata:name: opt-netapp-harvest-confnamespace: harvestannotations:volume.beta.kubernetes.io/storage-class: default-dburklandtrident.netapp.io/exportPolicy: "prod_kubernetes_nodes"trident.netapp.io/snapshotPolicy: "default-dburkland"spec:accessModes:- ReadWriteManyresources:requests:storage: 5Gi - Create the PVCs
12# kubectl create -f pvc-dburkland_harvest_carbon.yaml# kubectl create -f pvc-dburkland_harvest_conf.yaml - Define the deployment which relies on the aforementioned persistent volume claims
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283# vi setup/deployment-dburkland_harvest.yamlapiVersion: extensions/v1beta1kind: Deploymentmetadata:name: harvestnamespace: harvestspec:replicas: 1template:metadata:labels:app: harvestspec:volumes:- name: volume-var-lib-carbonpersistentVolumeClaim:claimName: var-lib-carbon- name: volume-opt-netapp-harvest-confpersistentVolumeClaim:claimName: opt-netapp-harvest-conf- name: volume-tmpfs-runemptyDir:medium: "Memory"- name: volume-tmpfs-run-lockemptyDir:medium: "Memory"- name: volume-tmpfs-tmpemptyDir:medium: "Memory"containers:- name: harvestimage: mn1s21000.dburkland.com:5000/dburkland/harvest:latestports:- containerPort: 80- containerPort: 8080volumeMounts:- mountPath: "/opt/netapp-harvest-conf"name: volume-opt-netapp-harvest-conf- mountPath: "/var/lib/carbon"name: volume-var-lib-carbon- mountPath: "/run"name: volume-tmpfs-run- mountPath: "/run/lock"name: volume-tmpfs-run-lock- mountPath: "/tmp"name: volume-tmpfs-tmpreadinessProbe:httpGet:path: /port: 80initialDelaySeconds: 120timeoutSeconds: 1periodSeconds: 15livenessProbe:httpGet:path: /port: 80initialDelaySeconds: 120timeoutSeconds: 1periodSeconds: 15securityContext:capabilities:add:- SYS_ADMIN---kind: Servicemetadata:name: harvestnamespace: harvestlabels:app: harvestspec:type: NodePortports:- name: grafanaport: 80nodePort: 30080- name: graphiteport: 8080nodePort: 30081selector:app: harvest - Create the deployment
1234# kubectl create -f setup/deployment-dburkland_harvest.yaml# kubectl get deployment --all-namespaces --watch=true<CTRL+C># kubectl get pod --all-namespaces --watch=true
- Login to your kubernetes master node and create the harvest namespace
- NetApp Controller / OCUM Appliance Preparation Steps
- Create the Harvest role on each NetApp system you wish to monitor
- 7mode
12useradmin role add role-harvest -c "Role for performance monitoring by NetApp Harvest" -a login-http-admin,api-system-get-version,api-system-get-info,api-perf-object-*,api-ems- autosupport-loguseradmin group add group-harvest -c "Group for performance monitoring by NetApp Harvest" -r role-harvest - cDOT
123456789security login role create -role role-harvest -access readonly -cmddirname "version"security login role create -role role-harvest -access readonly -cmddirname "cluster identity show"security login role create -role role-harvest -access readonly -cmddirname "cluster show"security login role create -role role-harvest -access readonly -cmddirname "system node show"security login role create -role role-harvest -access readonly -cmddirname "statistics"security login role create -role role-harvest -access readonly -cmddirname "lun show"security login role create -role role-harvest -access readonly -cmddirname "network interface show"security login role create -role role-harvest -access readonly -cmddirname "network interface show"security login role create -role role-harvest -access readonly -cmddirname "qos workload show"
- 7mode
- Create the Harvest user on each NetApp system you wish to monitor
- 7mode
1useradmin user add netapp-harvest -c "User account for performance monitoring by NetApp Harvest" -n "NetApp Harvest" -g group-harvest - cDOT 8.2 or earlier
1security login create -username harvest -application ontapi -role role-harvest -authmethod password - cDOT 8.3 or greater
1security login create -user-or-group-name harvest -application ontapi -role role-harvest -authmethod password - OCUM 6.x/7.x (Required for capacity data)
- For certificate authentication see the “NetApp Harvest Installation and Administration Guide” located here for more information
- 7mode
- Create the Harvest role on each NetApp system you wish to monitor
- Final NetApp Harvest Container Configuration & Validation Steps
- Login to the container and add all applicable NetApp 7mode or cDOT systems to the bottom of “/opt/netapp-harvest-conf/netapp-harvest.conf” and then restart the Harvest service to apply the changes
1234567891011121314151617# kubectl get pod --all-namespaces (Grab the harvest pod name)# kubectl exec -n harvest -ti harvest-1257702773-j1qzg -- /bin/bash# vi /opt/netapp-harvest-conf/netapp-harvest.conf[mn1d41000]host_type = FILERhostname = mn1d41000site = mn1[mn1s41007]host_type = OCUMhostname = mn1s41007.dburkland.comsite = mn1data_update_freq = 900# /etc/init.d/netapp-harvest restart# exit
- Login to the container and add all applicable NetApp 7mode or cDOT systems to the bottom of “/opt/netapp-harvest-conf/netapp-harvest.conf” and then restart the Harvest service to apply the changes
- Image download and distribution
- Sources
- [GitHub] dburkland/harvest Repository
- [GitHub] Tech ONTAP Live: NetApp Performance Tools Update Slide Deck
- NetApp Harvest Installation and Administration Guide
- Christopher Madden’s Blog
- 2258-3: Real-time Monitoring of Clustered Data ONTAP
- [DBurkland.com] How To Setup Graphite To Visualize OnCommand Performance Manager Data
- [DBurkland.com] How To Setup Grafana with Graphite And Create Basic Dashboards
- What is Docker?
- Docker Volumes Using NetApp Storage
- Deploy a registry server