Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Exploit IQ Self-Hosted Models
# RHTPA Exploit Intelligence Self-Hosted Models

## Overview

This repository contains an umbrella Helm chart for deploying AI models for the Exploit IQ platform. The chart deploys:
This repository contains an umbrella Helm chart for deploying AI models for the RHTPA exploit intelligence platform. The chart deploys:

- **Embedding model**: NVIDIA NIM embedding model (nv-embedqa-e5-v5) for creating vector embeddings
- **LLM**: One of the following large language models:
Expand Down Expand Up @@ -30,7 +30,7 @@ Before installing this chart, ensure you have:
Create a dedicated namespace for the models:

```bash
oc new-project exploit-iq-models
oc new-project exploit-intelligence-models
```

### Step 2: Prepare your NGC API key
Expand All @@ -49,8 +49,8 @@ Create a custom values file with your NGC API key:

```bash
sed -E 's/ \&ngc-api-key changeme/ \&ngc-api-key '$NGC_API_KEY'/' \
exploit-iq-models/values.yaml > \
exploit-iq-models/custom-values.yaml
exploit-intelligence-models/values.yaml > \
exploit-intelligence-models/custom-values.yaml
```

### Step 4: Configure tolerations (if needed)
Expand Down Expand Up @@ -80,25 +80,25 @@ nim-embed:
**Option A: Deploy with Llama 3.1 70B (default)**

```bash
helm upgrade --install exploit-iq-models \
exploit-iq-models/ \
-f exploit-iq-models/custom-values.yaml
helm upgrade --install exploit-intelligence-models \
exploit-intelligence-models/ \
-f exploit-intelligence-models/custom-values.yaml
```

**Option B: Deploy with NIM Llama 3.1 8B**

```bash
helm upgrade --install exploit-iq-models \
exploit-iq-models/ \
-f exploit-iq-models/custom-values.yaml \
helm upgrade --install exploit-intelligence-models \
exploit-intelligence-models/ \
-f exploit-intelligence-models/custom-values.yaml \
--set llama3_1_70b_instruct_4bit.enabled=false \
--set nim_llm.enabled=true
```

**Attempting to deploy both LLMs results in an error:**

```bash
Error: INSTALLATION FAILED: execution error at (exploit-iq-models/templates/configmap.yaml:6:3):
Error: INSTALLATION FAILED: execution error at (exploit-intelligence-models/templates/configmap.yaml:6:3):
Only one of models should be deployed!, either llama3_1_70b_instruct_4bit or nim_llm 8b, but not both!
```

Expand Down Expand Up @@ -128,7 +128,7 @@ Send a test request to the model:
```bash
curl -X POST -H "Content-Type: application/json" \
http://$ROUTE_URL/v1/chat/completions \
-d @exploit-iq-models/files/70b-4bit-input-example.json | jq .
-d @exploit-intelligence-models/files/70b-4bit-input-example.json | jq .
```

**Expected response**: JSON output with the model's completion.
Expand All @@ -139,7 +139,7 @@ curl -X POST -H "Content-Type: application/json" \

By default, all tolerations are empty arrays (`[]`), allowing the chart to work on clusters without GPU node taints. If your cluster uses taints to dedicate GPU nodes for specific workloads, configure tolerations in your values file.

See the comments in [exploit-iq-models/values.yaml](exploit-iq-models/values.yaml) for detailed examples.
See the comments in [exploit-intelligence-models/values.yaml](exploit-intelligence-models/values.yaml) for detailed examples.

### Security Context Constraints (SCC)

Expand Down Expand Up @@ -185,5 +185,5 @@ llama3_1_70b_instruct_4bit:
To remove the deployed models:

```bash
helm uninstall exploit-iq-models
helm uninstall exploit-intelligence-models
```
4 changes: 2 additions & 2 deletions exploit-iq-models/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
name: exploit-iq-models
description: An Umbrella Helm chart for Deploying ExploitIQ local LLMs and embedding models
name: exploit-intelligence-models
description: An Umbrella Helm chart for Deploying RHTPA exploit intelligence local LLMs and embedding models

# A chart can be either an 'application' or a 'library' chart.
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ helm.sh/chart: {{ include "llama3_1_70b_instruct_4bit.chart" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app: exploit-iq
app: exploit-intelligence
component: llama3.1-70b-instruct
inference: vllm
{{- end }}
Expand Down
4 changes: 2 additions & 2 deletions exploit-iq-models/charts/nim-embed/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -39,15 +39,15 @@ helm.sh/chart: {{ include "nim-embed.chart" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app: exploit-iq
app: exploit-intelligence
component: nim-embed
{{- end }}

{{/*
Selector labels
*/}}
{{- define "nim-embed.selectorLabels" -}}
app: exploit-iq
app: exploit-intelligence
component: nim-embed
{{- end }}

Expand Down
2 changes: 1 addition & 1 deletion exploit-iq-models/charts/nim_llm/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Selector labels
{{- define "nim_llm.selectorLabels" -}}
app.kubernetes.io/name: {{ include "nim_llm.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app: exploit-iq
app: exploit-intelligence
component: nim-llm
{{- end }}

Expand Down
24 changes: 12 additions & 12 deletions exploit-iq-models/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{{- define "exploit-iq-models.name" -}}
{{- define "exploit-intelligence-models.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}

{{- end }}

{{- define "exploit-iq-models.logic_check" -}}
{{- define "exploit-intelligence-models.logic_check" -}}
{{- if and .Values.llama3_1_70b_instruct_4bit.enabled .Values.nim_llm.enabled }}
hello: {{- required "Only one of models should be deployed!, either llama3_1_70b_instruct_4bit or nim_llm 8b, but not both!" .Values.whatever -}}
{{- end }}
Expand All @@ -16,7 +16,7 @@ Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "exploit-iq-models.fullname" -}}
{{- define "exploit-intelligence-models.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
Expand All @@ -32,38 +32,38 @@ If release name contains chart name it will be used as a full name.
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "exploit-iq-models.chart" -}}
{{- define "exploit-intelligence-models.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "exploit-iq-models.labels" -}}
helm.sh/chart: {{ include "exploit-iq-models.chart" . }}
{{ include "exploit-iq-models.selectorLabels" . }}
{{- define "exploit-intelligence-models.labels" -}}
helm.sh/chart: {{ include "exploit-intelligence-models.chart" . }}
{{ include "exploit-intelligence-models.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app: exploit-iq
app: exploit-intelligence
component: all-models
{{- end }}

{{/*
Selector labels
*/}}
{{- define "exploit-iq-models.selectorLabels" -}}
app.kubernetes.io/name: {{ include "exploit-iq-models.name" . }}
{{- define "exploit-intelligence-models.selectorLabels" -}}
app.kubernetes.io/name: {{ include "exploit-intelligence-models.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "exploit-iq-models.serviceAccountName" -}}
{{- define "exploit-intelligence-models.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "exploit-iq-models.fullname" .) .Values.serviceAccount.name }}
{{- default (include "exploit-intelligence-models.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
Expand Down
4 changes: 2 additions & 2 deletions exploit-iq-models/values.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Default values for exploit-iq-models.
# Default values for exploit-intelligence-models.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

global:
ngcAPIKey: &ngc-api-key changeme
serviceAccountName: &sa-name exploit-iq-models
serviceAccountName: &sa-name exploit-intelligence-models
# Toleration anchors removed to support clusters without GPU node taints.
# Configure tolerations for each subchart below if your cluster has tainted nodes.

Expand Down