本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。 # 使用 PersistentVolumeClaims (PVC) 對作業進行疑難排解 如果需要建立、列出或刪除作業的 PersistentVolumeClaims (PVC)，但未將 PVC 許可新增至預設 Kubernetes 角色 *emr-containers*，則當您提交作業時，作業會失敗。如果沒有這些許可，*emr-containers* 角色就無法為 Spark 驅動程式或 Spark 用戶端建立必要的角色。如錯誤訊息所建議，將許可新增至 Spark 驅動程式或用戶端角色是不夠的。*emr-containers* 主要角色也必須包含必要的許可。本章節說明如何將必要的許可新增至 *emr-containers* 主要角色。 ## 驗證 若要確認您的 *emr-containers* 角色是否擁有必要的許可，請使用自己的值來設定 NAMESPACE 變數，然後執行下列命令： ``` export NAMESPACE=YOUR_VALUE kubectl describe role emr-containers -n ${NAMESPACE} ``` 此外，若要驗證 Spark 和用戶端角色是否具有必要的許可，請執行下列命令： ``` kubectl describe role emr-containers-role-spark-driver -n ${NAMESPACE} kubectl describe role emr-containers-role-spark-client -n ${NAMESPACE} ``` 如果許可不存在，請繼續執行修補程式，如下所示。 ## 修補程式 1. 如果沒有許可的作業目前正在執行中，則請停止這些作業。 1. 建立一個名為 *RBAC\_Patch.py* 的檔案，如下所示： ``` import os import subprocess as sp import tempfile as temp import json import argparse import uuid def delete_if_exists(dictionary: dict, key: str): if dictionary.get(key, None) is not None: del dictionary[key] def doTerminalCmd(cmd): with temp.TemporaryFile() as f: process = sp.Popen(cmd, stdout=f, stderr=f) process.wait() f.seek(0) msg = f.read().decode() return msg def patchRole(roleName, namespace, extraRules, skipConfirmation=False): cmd = f"kubectl get role {roleName} -n {namespace} --output json".split(" ") msg = doTerminalCmd(cmd) if "(NotFound)" in msg and "Error" in msg: print(msg) return False role = json.loads(msg) rules = role["rules"] rulesToAssign = extraRules[::] passedRules = [] for rule in rules: apiGroups = set(rule["apiGroups"]) resources = set(rule["resources"]) verbs = set(rule["verbs"]) for extraRule in extraRules: passes = 0 apiGroupsExtra = set(extraRule["apiGroups"]) resourcesExtra = set(extraRule["resources"]) verbsExtra = set(extraRule["verbs"]) passes += len(apiGroupsExtra.intersection(apiGroups)) >= len(apiGroupsExtra) passes += len(resourcesExtra.intersection(resources)) >= len(resourcesExtra) passes += len(verbsExtra.intersection(verbs)) >= len(verbsExtra) if passes >= 3: if extraRule not in passedRules: passedRules.append(extraRule) if extraRule in rulesToAssign: rulesToAssign.remove(extraRule) break prompt_text = "Apply Changes?" if len(rulesToAssign) == 0: print(f"The role {roleName} seems to already have the necessary permissions!") prompt_text = "Proceed anyways?" for ruleToAssign in rulesToAssign: role["rules"].append(ruleToAssign) delete_if_exists(role, "creationTimestamp") delete_if_exists(role, "resourceVersion") delete_if_exists(role, "uid") new_role = json.dumps(role, indent=3) uid = uuid.uuid4() filename = f"Role-{roleName}-New_Permissions-{uid}-TemporaryFile.json" try: with open(filename, "w+") as f: f.write(new_role) f.flush() prompt = "y" if not skipConfirmation: prompt = input( doTerminalCmd(f"kubectl diff -f {filename}".split(" ")) + f"\n{prompt_text} y/n: " ).lower().strip() while prompt != "y" and prompt != "n": prompt = input("Please make a valid selection. y/n: ").lower().strip() if prompt == "y": print(doTerminalCmd(f"kubectl apply -f {filename}".split(" "))) except Exception as e: print(e) os.remove(f"./{filename}") if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument("-n", "--namespace", help="Namespace of the Role. By default its the VirtualCluster's namespace", required=True, dest="namespace" ) parser.add_argument("-p", "--no-prompt", help="Applies the patches without asking first", dest="no_prompt", default=False, action="store_true" ) args = parser.parse_args() emrRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch"] } ] driverRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch", "deletecollection"] }, { "apiGroups": [""], "resources": ["services"], "verbs": ["get", "list", "describe", "create", "delete", "watch", "deletecollection"] }, { "apiGroups": [""], "resources": ["configmaps", "pods"], "verbs": ["deletecollection"] } ] clientRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch"] } ] patchRole("emr-containers", args.namespace, emrRoleRules, args.no_prompt) patchRole("emr-containers-role-spark-driver", args.namespace, driverRoleRules, args.no_prompt) patchRole("emr-containers-role-spark-client", args.namespace, clientRoleRules, args.no_prompt) ``` 1. 執行 Python 指令碼： ``` python3 RBAC_Patch.py -n ${NAMESPACE} ``` 1. 新許可與舊許可之間的 kubectl 差異會出現。按 y 來修補角色。 1. 驗證具有其他許可的三個角色，如下所示： ``` kubectl describe role -n ${NAMESPACE} ``` 1. 執行 Python 指令碼： ``` python3 RBAC_Patch.py -n ${NAMESPACE} ``` 1. 執行命令後，它將顯示新許可和舊許可之間的 kubectl 差異。按 y 來修補角色。 1. 驗證具有其他許可的三個角色： ``` kubectl describe role -n ${NAMESPACE} ``` 1. 再次提交作業。 ## 手動修補 如果應用程式所需的許可適用於 PVC 規則以外的項目，可以視需要為 Amazon EMR 虛擬叢集手動新增 Kubernetes 許可。 **注意** *emr-containers* 角色是主要角色。這意味著它必須提供所有必要的許可，然後才能變更基礎驅動程式或用戶端角色。 1. 透過執行以下命令將當前許可下載到 yaml 檔案中： ``` kubectl get role -n ${NAMESPACE} emr-containers -o yaml >> emr-containers-role-patch.yaml kubectl get role -n ${NAMESPACE} emr-containers-role-spark-driver -o yaml >> driver-role-patch.yaml kubectl get role -n ${NAMESPACE} emr-containers-role-spark-client -o yaml >> client-role-patch.yaml ``` 1. 根據應用程式所需的許可，編輯每個檔案並新增其他規則，例如： + emr-containers-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch ``` + driver-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch - deletecollection - apiGroups: - "" resources: - services verbs: - get - list - describe - create - delete - watch - deletecollection - apiGroups: - "" resources: - configmaps - pods verbs: - deletecollection ``` + client-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch ``` 1. 移除下列屬性及其值。這對於套用更新是必要的。 + creationTimestamp + resourceVersion + uid 1. 最後，執行修補程式： ``` kubectl apply -f emr-containers-role-patch.yaml kubectl apply -f driver-role-patch.yaml kubectl apply -f client-role-patch.yaml ```