本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。 # 对使用 PersistentVolumeClaims (PVC) 的作业进行故障排除 如果你需要为任务创建、列出或删除 PersistentVolumeClaims (PVC)，但不向默认 Kubernetes 角色 *emr-* containers 添加 PVC 权限，那么当你提交任务时，任务就会失败。如果没有这些权限，则 *emr-containers* 角色无法为 Spark 驱动程序或 Spark 客户端创建必要的角色。如错误消息所示，向 Spark 驱动程序或客户端角色添加权限是不够的。*emr-containers* 主角色还必须包括所需的权限。本节介绍如何向 *emr-containers* 主角色添加所需的权限。 ## 验证 要验证您的 *emr-containers* 角色是否具有必要的权限，请使用您自己的值设置 NAMESPACE 变量，然后运行以下命令： ``` export NAMESPACE=YOUR_VALUE kubectl describe role emr-containers -n ${NAMESPACE} ``` 此外，若要验证 Spark 和客户端角色是否具有必要的权限，请运行以下命令： ``` kubectl describe role emr-containers-role-spark-driver -n ${NAMESPACE} kubectl describe role emr-containers-role-spark-client -n ${NAMESPACE} ``` 如果没有权限，请按如下方式继续修补。 ## 修补 1. 如果没有权限的作业当前正在运行，则停止这些作业。 1. 创建名为 *RBAC\_Patch.py* 的文件，如下所示： ``` import os import subprocess as sp import tempfile as temp import json import argparse import uuid def delete_if_exists(dictionary: dict, key: str): if dictionary.get(key, None) is not None: del dictionary[key] def doTerminalCmd(cmd): with temp.TemporaryFile() as f: process = sp.Popen(cmd, stdout=f, stderr=f) process.wait() f.seek(0) msg = f.read().decode() return msg def patchRole(roleName, namespace, extraRules, skipConfirmation=False): cmd = f"kubectl get role {roleName} -n {namespace} --output json".split(" ") msg = doTerminalCmd(cmd) if "(NotFound)" in msg and "Error" in msg: print(msg) return False role = json.loads(msg) rules = role["rules"] rulesToAssign = extraRules[::] passedRules = [] for rule in rules: apiGroups = set(rule["apiGroups"]) resources = set(rule["resources"]) verbs = set(rule["verbs"]) for extraRule in extraRules: passes = 0 apiGroupsExtra = set(extraRule["apiGroups"]) resourcesExtra = set(extraRule["resources"]) verbsExtra = set(extraRule["verbs"]) passes += len(apiGroupsExtra.intersection(apiGroups)) >= len(apiGroupsExtra) passes += len(resourcesExtra.intersection(resources)) >= len(resourcesExtra) passes += len(verbsExtra.intersection(verbs)) >= len(verbsExtra) if passes >= 3: if extraRule not in passedRules: passedRules.append(extraRule) if extraRule in rulesToAssign: rulesToAssign.remove(extraRule) break prompt_text = "Apply Changes?" if len(rulesToAssign) == 0: print(f"The role {roleName} seems to already have the necessary permissions!") prompt_text = "Proceed anyways?" for ruleToAssign in rulesToAssign: role["rules"].append(ruleToAssign) delete_if_exists(role, "creationTimestamp") delete_if_exists(role, "resourceVersion") delete_if_exists(role, "uid") new_role = json.dumps(role, indent=3) uid = uuid.uuid4() filename = f"Role-{roleName}-New_Permissions-{uid}-TemporaryFile.json" try: with open(filename, "w+") as f: f.write(new_role) f.flush() prompt = "y" if not skipConfirmation: prompt = input( doTerminalCmd(f"kubectl diff -f {filename}".split(" ")) + f"\n{prompt_text} y/n: " ).lower().strip() while prompt != "y" and prompt != "n": prompt = input("Please make a valid selection. y/n: ").lower().strip() if prompt == "y": print(doTerminalCmd(f"kubectl apply -f {filename}".split(" "))) except Exception as e: print(e) os.remove(f"./{filename}") if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument("-n", "--namespace", help="Namespace of the Role. By default its the VirtualCluster's namespace", required=True, dest="namespace" ) parser.add_argument("-p", "--no-prompt", help="Applies the patches without asking first", dest="no_prompt", default=False, action="store_true" ) args = parser.parse_args() emrRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch"] } ] driverRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch", "deletecollection"] }, { "apiGroups": [""], "resources": ["services"], "verbs": ["get", "list", "describe", "create", "delete", "watch", "deletecollection"] }, { "apiGroups": [""], "resources": ["configmaps", "pods"], "verbs": ["deletecollection"] } ] clientRoleRules = [ { "apiGroups": [""], "resources": ["persistentvolumeclaims"], "verbs": ["list", "create", "delete", "patch"] } ] patchRole("emr-containers", args.namespace, emrRoleRules, args.no_prompt) patchRole("emr-containers-role-spark-driver", args.namespace, driverRoleRules, args.no_prompt) patchRole("emr-containers-role-spark-client", args.namespace, clientRoleRules, args.no_prompt) ``` 1. 运行 Python 脚本： ``` python3 RBAC_Patch.py -n ${NAMESPACE} ``` 1. 显示新权限和旧权限之间的 kubectl 差异。按 y 以修补角色。 1. 使用附加权限验证三个角色，如下所示： ``` kubectl describe role -n ${NAMESPACE} ``` 1. 运行 Python 脚本： ``` python3 RBAC_Patch.py -n ${NAMESPACE} ``` 1. 运行命令后，将显示新旧权限之间的 kubectl 差异。按 y 以修补角色。 1. 使用附加权限验证三个角色： ``` kubectl describe role -n ${NAMESPACE} ``` 1. 再次提交作业。 ## 手动修补 如果您的应用程序所需的权限适用于 PVC 规则以外的其他内容，您可以根据需要为 Amazon EMR 虚拟集群手动添加 Kubernetes 权限。 **注意** 角色 *emr-containers* 是主要角色。这意味着它必须提供所有必要的权限，然后您才能更改底层驱动程序或客户端角色。 1. 通过运行以下命令将当前权限下载到 yaml 文件： ``` kubectl get role -n ${NAMESPACE} emr-containers -o yaml >> emr-containers-role-patch.yaml kubectl get role -n ${NAMESPACE} emr-containers-role-spark-driver -o yaml >> driver-role-patch.yaml kubectl get role -n ${NAMESPACE} emr-containers-role-spark-client -o yaml >> client-role-patch.yaml ``` 1. 根据应用程序所需的权限，编辑每个文件并添加其他规则，如下所示： + emr-containers-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch ``` + driver-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch - deletecollection - apiGroups: - "" resources: - services verbs: - get - list - describe - create - delete - watch - deletecollection - apiGroups: - "" resources: - configmaps - pods verbs: - deletecollection ``` + client-role-patch.yaml ``` - apiGroups: - "" resources: - persistentvolumeclaims verbs: - list - create - delete - patch ``` 1. 删除以下属性及其值。这是应用更新所必需的。 + creationTimestamp + resourceVersion + uid 1. 最后，运行补丁： ``` kubectl apply -f emr-containers-role-patch.yaml kubectl apply -f driver-role-patch.yaml kubectl apply -f client-role-patch.yaml ```