Summary
Overview & Analysis
The AI Safety Governance Framework 1.0 is a foundational Chinese policy document for governing AI safety across the full lifecycle of AI development and deployment. It is built around a "people-centered, AI for good" approach and explicitly stresses that development and security should be advanced together. Using a risk-management lens, it maps the risks that can arise during design, development, training, testing, deployment, use, and maintenance. The framework covers not only intrinsic AI risks such as poor interpretability, bias, weak robustness, tampering, and adversarial attacks, but also risks relating to training data, computing infrastructure, supply chains, and AI applications in the network, real-world, cognitive, and ethical domains. In that sense, it is not merely a document about content moderation or model safety; it is a broader framework for technical risk, application risk, and governance responsibility.
Its importance lies in the way it sets out China's policy logic for balancing AI innovation with safety governance. On the one hand, it supports AI innovation and application; on the other, it calls for mechanisms such as classified and tiered management, traceability of AI services, data and personal information protection, supply chain safeguards, interpretability research, emergency response, industry self-discipline, and public oversight. Legal analysis has generally treated the framework as a foundational technical-governance guide and a strong signal of regulatory direction: China is moving toward lifecycle-wide, risk-based, and multi-stakeholder AI governance.
Four Governance Principles
Principle I
Inclusiveness with Prudence & Ensuring Security
Principle II
Risk-Oriented, Agile Governance
Principle III
Integration of Technology & Management
Principle IV
Openness, Cooperation & Co-Governance
AI Risk Domains Covered
Intrinsic AI Risks
- Poor interpretability
- Bias & discrimination
- Weak robustness
- Theft & tampering
- Unreliable / hallucinated output
- Adversarial attacks
Data Security Risks
- Illegal data collection & use
- Training data poisoning
- Non-standard annotation
- Data leakage
- IP infringement in training data
System & Supply Chain Risks
- Defects & backdoors
- Computing power security
- Global supply chain disruption
- Cross-boundary compute-layer risks
Application Risks
- Misinformation & authentication bypass
- Cyberattack misuse
- Defect propagation via model reuse
- Cognitive warfare
- Social discrimination & loss of control
Relevant AI Scenarios
The framework is relevant in most major enterprise AI scenarios in China, especially where companies are developing models, preparing training data, offering AI services to the public, deploying AI in critical sectors, operating across borders, integrating third-party models, or managing higher-risk use cases. It assigns responsibilities to developers, service providers, key-sector users, and the public — making it relevant both to companies that build AI and those that procure, integrate, or operate it.
1. Developing, Training, Fine-Tuning, or Reusing Foundation Models
The framework directly addresses risks such as poor interpretability, bias, weak robustness, unreliable output, theft or tampering, and adversarial attack, and explicitly notes that defects in a foundation model can propagate downstream through reuse. Any company doing model development, fine-tuning, or secondary development in China should treat the framework as a key reference point for risk identification, testing, and responsibility allocation.
2. Preparing Training Data, Labeling Data, or Managing Data Flows
The framework stresses that training data and interaction data must comply with data-security and personal-information protection rules throughout collection, storage, use, transmission, disclosure, and deletion. It also highlights poisoning risk, leakage risk, non-standard labeling, unlawful sourcing, and intellectual property concerns. Data governance is not secondary to AI governance — it is central to it.
3. Providing Public-Facing AI Services, Content Generation, or Synthetic Media
The framework calls for a traceability management regime for AI systems serving the public and supports labeling requirements for AI-generated or synthesized content so that users can assess source and authenticity. Any public-facing AI service in China may need to consider product labeling, provenance controls, audit trails, and user communication measures to reduce risks of deception, impersonation, and bypass of authentication systems.
4. Deploying AI in Finance, Healthcare, Transport, Industrial Settings, or CII
The framework treats these as more sensitive deployment environments and expects prudent assessment, risk grading, regular audits, access control, backup and recovery planning, encryption, human supervision, and readiness to switch to manual systems. AI in high-impact settings cannot be judged only on efficiency or accuracy — it must also demonstrate safety, reliability, controllability, and emergency handling capability.
5. Cross-Border AI Services, Global Supply Chains, or Third-Party AI Procurement
The framework expressly states that cross-border provision of AI services must comply with China's cross-border data rules, that outbound provision of AI models and algorithms must comply with export-control requirements, and that supply chain risks around chips, software, tools, compute, and data resources must be monitored. China AI projects must be reviewed not only for model performance but also for data transfers, supply-chain resilience, third-party dependency, and traceable responsibility chains.
Practical Advice for Managers at Multinational Companies
AI compliance in China should not be treated only as a content-control issue or narrow data-compliance exercise. It should be treated as a broader governance topic covering models, data, systems, supply chains, application scenarios, user communication, and incident response. The key is a tiered, standardized, and traceable governance model so that low-risk projects move quickly while higher-risk ones are identified early and subjected to deeper review.
01
Move Safety Governance to Project Intake, Not Pre-Launch
Require each AI project to answer a small set of intake questions at the outset: Is it public-facing? Does it process personal information or important data? Does it affect high-impact decisions? Does it rely on a foundation model or third-party model? Is it used in a sensitive sector? Are there cross-border data or export-control implications? The earlier the company identifies the scenario, the less likely it is to face rework or delays later.
02
Build a "Tiered Governance + Fast/Slow Lane" Operating Model
Stratify projects by risk. Low-risk internal productivity tools can use light-touch review templates and fast approval; medium-risk projects should involve joint review by business, data, legal, and information security teams; high-risk projects should require stronger testing, logging, explainability, human intervention, rollback design, and senior sign-off. The framework expressly calls for classified and tiered management of AI applications.
03
Treat Data Governance as the Foundation of AI Governance
Push data questions to the beginning of product and model development. Teams should be able to explain where training and inference data comes from, whether it is authentic, accurate, objective, diverse, and lawfully sourced, whether it includes sensitive personal information or important data, whether there are intellectual property issues, and whether it could introduce bias or poisoning risk. Many China AI projects stall not because of model design, but because data provenance is unclear.
04
Conduct "Responsibility Chain" Review for Third-Party Models & Vendors
The framework specifically says service providers should examine responsibility documentation from developers and ensure that responsibility can be traced through recursively adopted AI models. AI procurement cannot be reduced to functionality and price. Ask who built the underlying model, what the training data and limitations are, how security flaws are remediated, how outputs are labeled, and who is accountable if something goes wrong. Responsibility does not disappear because a third-party model is used.
05
Put Transparency, Labeling & Explainability into Product Design
The framework calls for appropriate disclosure of principles, capabilities, applicable scenarios, and safety risks, clear labeling of outputs, and explanation plans where AI decisions have major impacts. When is the user told they are dealing with AI? Are synthetic outputs marked? Can important outputs be explained in plain terms? Can internal teams quickly explain why a system produced a certain result? This both meets regulatory expectations and reduces user confusion and complaint risk.
06
In Key Sectors, Insist on Human Control and Manual Fallback
For healthcare, transport, industrial operations, public services, government, and critical information infrastructure, the framework strongly emphasizes human authorization, human supervision, and the ability to switch to manual or traditional systems in time. Treat this as an operational requirement: critical systems should have defined human approval points, shutdown rights, fallback procedures, escalation paths, and drills.
07
Integrate Cybersecurity, Supply-Chain Security & AI Safety into One View
The framework expressly mentions chips, software, tools, compute resources, data resources, vulnerabilities, backdoors, cross-boundary compute-layer risks, and supply disruption. China AI governance should not sit with an isolated "AI lead" alone — it should involve AI teams, the CISO function, procurement, legal, data governance, and business owners together, using one shared risk map and escalation structure.
08
Build Incident Response and Reporting Readiness Before Launch
The framework calls for risk-threat information sharing and emergency response mechanisms for AI security incidents, and says service providers should promptly report security incidents and vulnerabilities. AI systems should launch with log retention, risk monitoring, alert thresholds, shutdown authority, internal escalation matrices, and external reporting paths already defined. Effective governance means the ability to detect, contain, fix, and document the company's response quickly and credibly.
09
Keep Global Principles, but Localize the China Implementation Layer
Many multinationals have global AI principles, but China's framework is more specific on traceability, obligations for users in key sectors, cross-border AI services, model responsibility chains, and supply-chain security. Add a China implementation annex under the global framework: define which projects require local review, which scenarios require Chinese-language documentation, which vendor clauses are mandatory, and which China-facing features need labeling, restrictions, or human review.
The most practical value of this framework is not that it gives a single rigid blacklist of prohibited conduct, but that it offers an operating logic: manage AI through risk, combine technical and managerial controls, connect the responsibilities of developers, providers, users, and oversight actors, and treat model security, data security, system security, and application security as one governance structure. When those principles are turned into project tiering, data review, vendor admission standards, product labeling, human control design, and incident response processes, companies can scale AI in China more confidently without forcing a false choice between speed and compliance.
Full Text
Complete Framework Text
Document Type: Governance framework — issued by the National Technical Committee for Cybersecurity Standardization. This document is a policy-level technical governance framework and not a standalone binding statute. It functions as a recognized operational baseline for AI safety governance in China and provides direction for sector-specific regulation, procurement assessment, and product design.
Table of Contents
- Article 1 — Principles of AI Security Governance
- Article 2 — Structure of the AI Security Governance Framework
- Article 3 — Classification of AI Security Risks (Intrinsic)
- Article 4 (Part I) — AI Application Security Risks
- Article 4 (Part II) — Technical Countermeasures
- Article 5 — Comprehensive Governance Measures (10 measures)
- Article 6 — Guidelines for Safe Development & Application
Article 1 — Principles of AI Security Governance
Upholding a security concept that is common, comprehensive, cooperative, and sustainable, adhering to equal emphasis on development and security, taking the promotion of AI innovation and development as the primary task, and taking the effective prevention and mitigation of AI security risks as the starting point and objective, a governance mechanism is established featuring multi-stakeholder participation, integration of technical and managerial approaches, and division of responsibilities and collaboration. Responsibilities for security are clarified and enforced for relevant stakeholders, a full-process and full-factor governance chain is constructed, and a safe, reliable, fair, and transparent AI technology development and application ecosystem is cultivated.
Principle 1 — Inclusiveness with Prudence, Ensuring Security
Encourage development and innovation, adopting an inclusive attitude toward AI R&D and applications. Strictly uphold security baselines and promptly take measures against risks that endanger national security, public interests, or the lawful rights and interests of the public.
Principle 2 — Risk-Oriented, Agile Governance
Closely track trends in AI R&D and applications, analyze and identify security risks from both AI technologies themselves and their applications, and propose targeted prevention and response measures. Monitor changes in security risks, dynamically and precisely adjust governance measures, continuously optimize governance mechanisms and methods, and respond promptly to matters requiring government regulation.
Principle 3 — Integration of Technology and Management, Coordinated Response
Across the entire lifecycle of AI R&D and application, comprehensively apply combined technical and managerial governance measures to prevent and address various types of security risks. Clarify the security responsibilities of model and algorithm developers, service providers, users, and other stakeholders across the AI ecosystem, and effectively leverage governance mechanisms such as government regulation, industry self-discipline, and social supervision.
Principle 4 — Openness and Cooperation, Co-Governance and Sharing
Promote international cooperation in AI security governance globally, share best practices, advocate the establishment of open platforms, and through cross-disciplinary, cross-sector, cross-regional, and cross-border dialogue and collaboration, facilitate the formation of a widely recognized global AI governance system.
Article 2 — Structure of the AI Security Governance Framework
Based on a risk management approach, this framework proposes preventive and response measures from both technical and managerial perspectives for different types of AI security risks. As AI R&D and applications continue to evolve rapidly, the forms, impacts, and perceptions of security risks will also change accordingly. Preventive and response measures will therefore be dynamically adjusted and updated, requiring all stakeholders to continuously optimize and improve the governance framework.
Dimension 1 — Security Risk
By analyzing the characteristics of AI technologies and their application scenarios across different industries, identify and categorize various security risks inherent in AI technologies and those arising during application.
Dimension 2 — Technical Countermeasures
For models and algorithms, training data, computing infrastructure, products and services, and application scenarios, propose technical measures — such as secure software development, data quality improvement, secure construction and operation, evaluation, monitoring, and reinforcement — to enhance the safety, fairness, reliability, and robustness of AI products and applications.
Dimension 3 — Comprehensive Governance Measures
Clarify measures for identifying, preventing, and responding to AI security risks among stakeholders including R&D institutions, service providers, users, government departments, industry associations, and social organizations, and promote coordinated governance among all parties.
Dimension 4 — Guidelines for Secure Development and Application
Define a set of security guidelines for the development and application of AI technologies for model and algorithm developers, service providers, key sector users, and the general public.
Article 3 — Classification of AI Security Risks (Intrinsic)
AI systems face security risks throughout their lifecycle, including design, development, training, testing, deployment, use, and maintenance. These risks arise both from inherent technical defects and limitations, as well as from improper use, misuse, or even malicious exploitation.
1. Model and Algorithm Security Risks
Risk 1 — Poor Interpretability
AI algorithms, particularly deep learning models, have complex internal operational logic, with inference processes functioning as "black box" or "gray box" models. This may result in outputs that are difficult to predict or attribute precisely, making it challenging to quickly correct anomalies or trace accountability.
Risk 2 — Bias and Discrimination
During algorithm design and training, individual biases may be intentionally or unintentionally introduced, or due to issues in training dataset quality, leading to biased or discriminatory design objectives and outputs, including content reflecting discrimination based on ethnicity, religion, nationality, or region.
Risk 3 — Weak Robustness
Due to the nonlinear and large-scale nature of deep neural networks, AI systems are susceptible to complex and changing operating environments or malicious interference and manipulation, potentially leading to performance degradation, incorrect decisions, and other issues.
Risk 4 — Theft and Tampering
Core algorithmic information such as parameters, structures, and functions may be subject to reverse engineering attacks, theft, modification, or even the insertion of backdoors. This may result in intellectual property infringement, leakage of trade secrets, unreliable inference processes, incorrect decision outputs, or system malfunctions.
Risk 5 — Unreliable Output (Hallucination)
Generative AI may produce "hallucinations," generating content that appears reasonable but is actually inconsistent with reality, leading to knowledge bias and misinformation.
Risk 6 — Adversarial Attacks
Attackers may create carefully designed adversarial samples to subtly mislead, influence, or manipulate AI models, causing incorrect outputs or even system failure.
2. Data Security Risks
Risk 7 — Illegal Data Collection and Use
During the acquisition of AI training data and interactions between services and users, there is a risk of collecting data and personal information without consent or using them improperly.
Risk 8 — Improper Content or "Poisoned" Training Data
Training data may contain illegal or harmful information such as falsehoods, biases, or intellectual property violations, or lack diversity, leading to harmful outputs. Training data may also be subject to "poisoning" attacks involving tampering, injection of false data, or misleading inputs, contaminating the model's probability distribution and reducing accuracy and reliability.
Risk 9 — Non-Standard Data Annotation
During data labeling, issues such as incomplete annotation rules, insufficient annotator capability, or labeling errors may arise, affecting model accuracy, reliability, and effectiveness, and potentially leading to bias amplification, reduced generalization capability, or incorrect outputs.
Risk 10 — Data Leakage
During AI R&D and application, improper data handling, unauthorized access, malicious attacks, or induced interactions may result in data and personal information leakage.
3. System Security Risks
Risk 11 — Exploitation of Defects and Backdoors
Standard interfaces, feature libraries, toolkits, development interfaces, and execution platforms used in AI model design, training, and validation may contain logical defects or vulnerabilities, and may be maliciously implanted with backdoors, posing risks of triggering and exploitation.
Risk 12 — Computing Power Security
AI training and operation rely on computing infrastructure involving multi-source and ubiquitous computing nodes and diverse computing resources, which face risks such as malicious consumption of computing resources and cross-boundary transmission of risks at the computing layer.
Risk 13 — Supply Chain Security
The AI industry chain is highly globalized and collaborative. However, certain countries impose unilateral coercive measures such as technological monopolies and export controls to create barriers to development, maliciously disrupting the global AI supply chain and causing risks such as shortages of chips, software, and tools.
Article 4 (Part I) — AI Application Security Risks
Network Domain Security Risks
Information Content Security
Content generated or synthesized by AI may easily lead to the spread of false information, discrimination and bias, privacy leakage, and infringement, posing threats to the safety of citizens' lives and property, national security, ideological security, and ethical security. If user prompts contain inappropriate content and the model's safety protection mechanisms are inadequate, it may output illegal or harmful content.
Confusing Facts, Misleading Users & Bypassing Authentication
Unlabeled AI systems and outputs make it difficult for users to identify whether the interaction object or generated content originates from an AI system, and to determine the authenticity of the content, thereby affecting user judgment and causing misunderstanding. At the same time, highly realistic AI-generated images, audio, and video may bypass existing identity authentication mechanisms such as facial recognition and voice recognition, leading to authentication failures.
Information Leakage Due to Improper Use
Government and enterprise personnel may improperly or irregularly use AI services in their work, inputting internal business data or industrial information into large models, resulting in leakage of work secrets, trade secrets, and sensitive business data.
Misuse for Cyber Attacks
Artificial intelligence may be used to conduct automated cyberattacks or improve attack efficiency, including exploiting vulnerabilities, cracking passwords, generating malicious code, sending phishing emails, network scanning, and social engineering attacks, thereby lowering the threshold for cyberattacks.
Defect Propagation Through Model Reuse
Secondary development or fine-tuning based on foundational models is a common AI application approach. If the foundational model contains security defects, the risks will propagate to downstream models.
Real-World Domain Security Risks
Traditional Economic and Social Security Issues
When AI is applied in finance, energy, telecommunications, transportation, and public services — such as autonomous driving and intelligent diagnosis — hallucinated outputs, incorrect decisions, and system performance degradation or loss of control caused by improper use or external attacks may pose threats to personal safety, property, and economic and social stability.
Use in Illegal and Criminal Activities
Artificial intelligence may be exploited in traditional illegal and criminal activities involving terrorism, violence, gambling, and drugs, including teaching criminal techniques, concealing illegal activities, and producing tools for criminal acts.
Misuse of Dual-Use Items and Technologies
Improper use or misuse of AI dual-use items and technologies may significantly lower the threshold for non-experts to design, synthesize, acquire, and use nuclear, biological, chemical, or missile weapons, and may enable automated cyberattacks against a wide range of targets.
Cognitive Domain Security Risks
The "Information Cocoon" Effect
Artificial intelligence will be widely applied in personalized information services, collecting user data and analyzing user types, needs, intentions, preferences, behavioral habits, and even prevailing public opinions during specific periods, thereby pushing templated and customized information and services, further intensifying the "information cocoon" effect.
Use in Cognitive Warfare
Artificial intelligence may be used to create and disseminate false news, images, audio, and videos, promote terrorism, extremism, and organized crime, interfere with other countries' internal affairs, and harm their sovereignty. Through social bots, it may seize discourse power and agenda-setting power in cyberspace, influencing public values and cognition.
Ethical Domain Security Risks
Social Discrimination and the Digital Divide
By using AI to collect and analyze human behavior, social status, economic conditions, and individual characteristics, different groups may be categorized and treated differently, leading to systemic and structural discrimination and bias. At the same time, the AI gap between regions may widen.
Challenging Traditional Social Order
The development and application of AI may bring significant changes to production tools and production relations, accelerate the restructuring of traditional industry models, and disrupt traditional views on employment, reproduction, and education, posing challenges to the stable operation of traditional social order.
Risk of Future Loss of Control
With the rapid development of AI technology, it cannot be ruled out that AI may autonomously acquire external resources, replicate itself, develop self-awareness, seek external power, and pose risks of competing with humans for control.
Article 4 (Part II) — Technical Countermeasures
Countermeasures for Intrinsic AI Security Risks
Model & Algorithm Countermeasures
(a) Continuously improve the interpretability and predictability of AI, providing clear explanations of internal structures, inference logic, technical interfaces, and outputs, accurately reflecting how results are generated.
(b) Establish and implement secure development standards during design, development, deployment, and maintenance to eliminate security defects and discriminatory tendencies in models and algorithms as much as possible and improve robustness.
(b) Establish and implement secure development standards during design, development, deployment, and maintenance to eliminate security defects and discriminatory tendencies in models and algorithms as much as possible and improve robustness.
Data Security Countermeasures
(a) Throughout all stages of data handling — including collection, storage, use, processing, transmission, provision, disclosure, and deletion of training data and user interaction data — adhere to data security and personal information protection rules, and strictly implement legal rights such as user control, right to know, and right to choose.
(b) Strengthen intellectual property protection, preventing infringement during training data selection and output generation.
(c) Strictly screen training data to ensure that it does not contain sensitive data related to high-risk areas such as nuclear, biological, chemical, or missile weapons.
(d) If training data contains sensitive personal information or important data, strengthen data security management and comply with relevant standards and regulations.
(e) Use training data that is authentic, accurate, objective, diverse, and lawfully sourced, and promptly filter out invalid, erroneous, or biased data.
(f) Provision of AI services across borders shall comply with cross-border data regulations, and provision of AI models and algorithms abroad shall comply with export control requirements.
(b) Strengthen intellectual property protection, preventing infringement during training data selection and output generation.
(c) Strictly screen training data to ensure that it does not contain sensitive data related to high-risk areas such as nuclear, biological, chemical, or missile weapons.
(d) If training data contains sensitive personal information or important data, strengthen data security management and comply with relevant standards and regulations.
(e) Use training data that is authentic, accurate, objective, diverse, and lawfully sourced, and promptly filter out invalid, erroneous, or biased data.
(f) Provision of AI services across borders shall comply with cross-border data regulations, and provision of AI models and algorithms abroad shall comply with export control requirements.
System Security Countermeasures
(a) Appropriately disclose the principles, capabilities, applicable scenarios, and security risks of AI technologies and products, clearly label outputs, and continuously improve system transparency.
(b) For platforms aggregating multiple AI models or systems, strengthen risk identification, detection, and protection to prevent malicious behavior or attacks from affecting hosted systems.
(c) Strengthen the security construction, management, and operation of AI computing platforms and system services, ensuring uninterrupted operation of infrastructure and services.
(d) Closely monitor supply chain security for chips, software, tools, computing power, and data resources; track vulnerabilities and defects in hardware and software; and promptly implement patches and reinforcement measures to ensure system security.
(b) For platforms aggregating multiple AI models or systems, strengthen risk identification, detection, and protection to prevent malicious behavior or attacks from affecting hosted systems.
(c) Strengthen the security construction, management, and operation of AI computing platforms and system services, ensuring uninterrupted operation of infrastructure and services.
(d) Closely monitor supply chain security for chips, software, tools, computing power, and data resources; track vulnerabilities and defects in hardware and software; and promptly implement patches and reinforcement measures to ensure system security.
Countermeasures for AI Application Security Risks
Network Domain Countermeasures
(a) Establish security protection mechanisms to prevent interference or tampering during model operation that could lead to unreliable outputs.
(b) Establish data safeguards to ensure that AI system outputs involving sensitive personal information and important data comply with relevant laws and regulations.
(b) Establish data safeguards to ensure that AI system outputs involving sensitive personal information and important data comply with relevant laws and regulations.
Real-World Domain Countermeasures
(a) Define service boundaries based on actual application scenarios, limit functionalities that may be misused, and ensure services do not exceed predefined application scopes.
(b) Enhance traceability of AI system end uses to prevent misuse in high-risk scenarios such as the development of weapons of mass destruction.
(b) Enhance traceability of AI system end uses to prevent misuse in high-risk scenarios such as the development of weapons of mass destruction.
Cognitive & Ethical Domain Countermeasures
(a) Use technical means to identify outputs that do not meet expectations or are untrue or inaccurate, and regulate them in accordance with laws and regulations.
(b) Strictly prevent misuse of AI systems that analyze user queries to infer identity, preferences, or ideological tendencies.
(c) Strengthen research and development of detection technologies for AI-generated synthetic content, enhancing capabilities to prevent, detect, and respond to cognitive warfare tactics.
(d) In algorithm design, model training and optimization, and service provision, methods such as training data screening and output validation shall be adopted to prevent discrimination based on ethnicity, belief, nationality, region, gender, age, occupation, health, and other factors.
(e) AI systems applied in key sectors — such as government departments, critical information infrastructure, and fields that directly affect public safety and the life and health safety of citizens — shall possess efficient and precise emergency control measures.
(b) Strictly prevent misuse of AI systems that analyze user queries to infer identity, preferences, or ideological tendencies.
(c) Strengthen research and development of detection technologies for AI-generated synthetic content, enhancing capabilities to prevent, detect, and respond to cognitive warfare tactics.
(d) In algorithm design, model training and optimization, and service provision, methods such as training data screening and output validation shall be adopted to prevent discrimination based on ethnicity, belief, nationality, region, gender, age, occupation, health, and other factors.
(e) AI systems applied in key sectors — such as government departments, critical information infrastructure, and fields that directly affect public safety and the life and health safety of citizens — shall possess efficient and precise emergency control measures.
Article 5 — Comprehensive Governance Measures
Measure 1 — Classified & Tiered Management of AI Applications
According to functions, performance, application scenarios, and other factors, AI systems shall be classified and graded, and a risk-level testing and evaluation system shall be established. End-use management of AI shall be strengthened, and relevant requirements shall be imposed on the use of AI technologies by specific groups of people and in specific scenarios, in order to prevent the misuse of AI systems. AI systems whose computing power or inference capabilities reach certain thresholds, or which are applied in specific industries and sectors, shall be registered and filed.
Measure 2 — Traceability Management System for AI Services
For AI systems providing services to the public, digital certificate technology shall be used to conduct identification management. Standards and specifications for labeling AI-generated and synthetic content shall be formulated and issued, making explicit requirements for visible, invisible, and other forms of labeling, comprehensively covering key links such as the source of production, transmission paths, and distribution channels, so as to facilitate users in identifying and judging information sources and authenticity.
Measure 3 — AI Data Security & Personal Information Protection Rules
In light of the characteristics of AI technologies and applications, clarify the requirements for data security and personal information protection at all stages of AI training, labeling, use, and output.
Measure 4 — Responsible AI R&D and Application System
Study and propose specific operational guidelines and best practices for implementing the principles of "people-centered, AI for good" in AI R&D and application, and continuously promote the alignment of values and ethics in AI design, R&D, and application. Explore copyright protection and utilization systems suited to the AI era, continue advancing the construction of high-quality foundational corpora and datasets, and provide high-quality inputs for the secure development of AI. Formulate AI ethics review principles, norms, and guidelines, and improve the ethics review system.
Measure 5 — AI Supply Chain Security Safeguards
Promote the sharing of AI knowledge achievements, open-source AI technologies, and jointly develop AI chips, frameworks, and software; guide the industry in establishing an open ecosystem; enhance the diversity of supply chain sources; and ensure the security and stability of the AI supply chain.
Measure 6 — AI Interpretability Research
Organize research from the perspectives of machine learning theory, training methods, and human-computer interaction on issues such as the transparency, trustworthiness, and error-correction mechanisms of AI decision-making; continuously improve the interpretability and predictability of AI; and avoid malicious behavior arising from unexpected AI decisions.
Measure 7 — AI Security Risk Information Sharing & Emergency Response
Continuously track and analyze trends relating to security vulnerabilities, defects, risk threats, and security incidents involving AI technologies, hardware and software products, and services, and coordinate relevant developers and service providers to establish mechanisms for reporting and sharing information on risks and threats. Build emergency response mechanisms for AI security incidents, formulate contingency plans, carry out emergency drills, and promptly, rapidly, and effectively handle AI security threats and incidents.
Measure 8 — AI Security Talent Cultivation
Promote the synchronized development of AI security education and AI disciplines; rely on schools, research institutions, and others to strengthen the cultivation of talent in AI security design, development, and governance; support the cultivation of top talent in frontier foundational fields of AI security; and strengthen security talent teams in areas such as autonomous driving, intelligent healthcare, brain-inspired intelligence, and brain-computer interfaces.
Measure 9 — AI Security Publicity, Self-Discipline & Social Supervision
Strengthen education and training on the standardized and secure application of AI for government, enterprises, and public utilities. Guide and support industry associations to strengthen industry self-discipline, formulate AI security self-regulatory conventions that exceed regulatory requirements, and guide and urge AI technology R&D institutions and service providers to continuously improve their security capabilities. Establish complaint and reporting acceptance mechanisms for AI security risks for the public, thereby creating an effective atmosphere of social supervision over AI security.
Measure 10 — International Exchanges & Cooperation in AI Security Governance
Actively conduct cooperation and exchanges with all countries on AI, support the establishment of an international AI governance body within the United Nations framework, and coordinate major issues concerning AI development, security, and governance. Advance cooperation on AI security governance under multilateral mechanisms such as APEC, the G20, and BRICS; strengthen cooperation with countries participating in the Belt and Road Initiative and countries of the "Global South"; study the establishment of an AI security governance alliance; and enhance the representation and voice of developing countries in global AI governance.
Article 6 — Guidelines for Safe Development and Application
Part A — Guidelines for Model and Algorithm Developers
Developer Guidelines (10 Points)
(1) In key stages such as requirements analysis, project initiation, model design and development, and training data selection, earnestly practice the principle of "people-centered, AI for good," follow scientific and technological ethics norms, and adopt measures such as conducting internal discussions, organizing expert reviews, carrying out science and technology ethics reviews, soliciting public opinions, communicating with potential target users, and strengthening employee security education and training.
(2) In contracts or service agreements, inform users of the scope of application, precautions, and contraindications of AI products and services in a manner easily understandable to users, and support users in making informed choices and using them prudently.
(3) In informed consent documents and service agreements, support users in exercising responsibilities for human supervision and control.
(4) Enable users to understand the accuracy of AI products, and, where AI decisions have a major impact, prepare plans for explanation and clarification.
(5) Examine the responsibility documentation provided by developers to ensure that the chain of responsibility can be traced back to recursively adopted AI models.
(6) Establish and improve real-time risk monitoring and management mechanisms, and continuously track security risks during operation.
(7) Assess the ability of AI products and services to resist or overcome adverse conditions when facing abnormal conditions such as failures and attacks, prevent unexpected results and behavioral errors, and ensure a minimum level of effective functionality.
(8) Promptly report security incidents, security vulnerabilities, and the like discovered during the operation of AI systems to the competent authorities.
(9) Clearly stipulate in contracts or service agreements that, once misuse or abuse inconsistent with the intended use and stated limitations is discovered, the service provider has the right to take corrective measures or terminate the service in advance.
(10) Assess the impact of AI products on users, and prevent harm to users' physical and mental health, life, property, and other interests.
(2) In contracts or service agreements, inform users of the scope of application, precautions, and contraindications of AI products and services in a manner easily understandable to users, and support users in making informed choices and using them prudently.
(3) In informed consent documents and service agreements, support users in exercising responsibilities for human supervision and control.
(4) Enable users to understand the accuracy of AI products, and, where AI decisions have a major impact, prepare plans for explanation and clarification.
(5) Examine the responsibility documentation provided by developers to ensure that the chain of responsibility can be traced back to recursively adopted AI models.
(6) Establish and improve real-time risk monitoring and management mechanisms, and continuously track security risks during operation.
(7) Assess the ability of AI products and services to resist or overcome adverse conditions when facing abnormal conditions such as failures and attacks, prevent unexpected results and behavioral errors, and ensure a minimum level of effective functionality.
(8) Promptly report security incidents, security vulnerabilities, and the like discovered during the operation of AI systems to the competent authorities.
(9) Clearly stipulate in contracts or service agreements that, once misuse or abuse inconsistent with the intended use and stated limitations is discovered, the service provider has the right to take corrective measures or terminate the service in advance.
(10) Assess the impact of AI products on users, and prevent harm to users' physical and mental health, life, property, and other interests.
Part B — Guidelines for Key Sector Users
Key Sector User Guidelines (9 Points)
(1) Prudently assess the long-term and potential impacts brought by the adoption of AI technologies in target application scenarios, carry out risk assessment and grading, and avoid technological misuse.
(2) Based on the applicable scenarios, safety, reliability, and controllability of AI systems, regularly conduct system audits and strengthen awareness of risk prevention and capacity for risk response.
(3) Before using AI products, comprehensively understand their data processing and privacy protection measures.
(4) Use high-security-level password strategies, enable multi-factor authentication mechanisms, and enhance account security.
(5) Strengthen capabilities in cybersecurity and supply chain security, reduce the risks of AI systems being attacked and important data being stolen or leaked, and ensure uninterrupted business operations.
(6) Reasonably restrict AI systems' access permissions to data, formulate data backup and recovery plans, and regularly inspect data processing workflows.
(7) Ensure that operations comply with confidentiality requirements and use protective measures such as encryption technologies when processing sensitive data.
(8) Effectively supervise AI behaviors and impacts, ensuring that the operation of AI products and services is based on human authorization and remains under human control.
(9) Avoid complete reliance on AI decisions, monitor and record situations in which AI decisions are not adopted, analyze inconsistencies in decision-making, and possess the ability to switch in a timely manner to manual or traditional systems in the event of accidents.
(2) Based on the applicable scenarios, safety, reliability, and controllability of AI systems, regularly conduct system audits and strengthen awareness of risk prevention and capacity for risk response.
(3) Before using AI products, comprehensively understand their data processing and privacy protection measures.
(4) Use high-security-level password strategies, enable multi-factor authentication mechanisms, and enhance account security.
(5) Strengthen capabilities in cybersecurity and supply chain security, reduce the risks of AI systems being attacked and important data being stolen or leaked, and ensure uninterrupted business operations.
(6) Reasonably restrict AI systems' access permissions to data, formulate data backup and recovery plans, and regularly inspect data processing workflows.
(7) Ensure that operations comply with confidentiality requirements and use protective measures such as encryption technologies when processing sensitive data.
(8) Effectively supervise AI behaviors and impacts, ensuring that the operation of AI products and services is based on human authorization and remains under human control.
(9) Avoid complete reliance on AI decisions, monitor and record situations in which AI decisions are not adopted, analyze inconsistencies in decision-making, and possess the ability to switch in a timely manner to manual or traditional systems in the event of accidents.
Part C — Guidelines for the General Public
General Public Guidelines (6 Points)
(1) Improve awareness of the security risks of AI products and choose AI products with good reputations.
(2) Before use, carefully read product contracts or service agreements, understand the functions, limitations, and privacy policies of the products, accurately recognize the limitations of AI products in making judgments and decisions, and reasonably set expectations for use.
(3) Enhance awareness of personal information protection and avoid inputting sensitive information where unnecessary.
(4) Understand the data processing methods of AI products and avoid using products that do not comply with privacy protection principles.
(5) When using AI products, pay attention to cybersecurity risks and avoid allowing AI products to become targets of cyberattacks.
(6) Pay attention to the impact of AI products on children and adolescents, and prevent addiction and excessive use.
(2) Before use, carefully read product contracts or service agreements, understand the functions, limitations, and privacy policies of the products, accurately recognize the limitations of AI products in making judgments and decisions, and reasonably set expectations for use.
(3) Enhance awareness of personal information protection and avoid inputting sensitive information where unnecessary.
(4) Understand the data processing methods of AI products and avoid using products that do not comply with privacy protection principles.
(5) When using AI products, pay attention to cybersecurity risks and avoid allowing AI products to become targets of cyberattacks.
(6) Pay attention to the impact of AI products on children and adolescents, and prevent addiction and excessive use.
人工智能安全治理框架
全国网络安全标准化技术委员会，2024年9月
目 录
第一条 人工智能安全治理原则
秉持共同、综合、合作、可持续的安全观，坚持发展和安全并重，以促进人工智能创新发展为第一要务，以有效防范化解人工智能安全风险为出发点和落脚点，构建各方共同参与、技管结合、分工协作的治理机制，压实相关主体安全责任，打造全过程全要素治理链条，培育安全、可靠、公平、透明的人工智能技术研发和应用生态，推动人工智能健康发展和规范应用，切实维护国家主权、安全和发展利益，保障公民、法人和其他组织的合法权益，确保人工智能技术造福于人类。
原则一 — 包容审慎、确保安全
鼓励发展创新，对人工智能研发及应用采取包容态度。严守安全底线，对危害国家安全、社会公共利益、公众合法权益的风险及时采取措施。
原则二 — 风险导向、敏捷治理
密切跟踪人工智能研发及应用趋势，从人工智能技术自身、人工智能应用两方面分析梳理安全风险，提出针对性防范应对措施。关注安全风险发展变化，快速动态精准调整治理措施，持续优化治理机制和方式，对确需政府监管事项及时予以响应。
原则三 — 技管结合、协同应对
面向人工智能研发应用全过程，综合运用技术、管理相结合的安全治理措施，防范应对不同类型安全风险。围绕人工智能研发应用生态链，明确模型算法研发者、服务提供者、使用者等相关主体的安全责任，有机发挥政府监管、行业自律、社会监督等治理机制作用。
原则四 — 开放合作、共治共享
在全球范围推动人工智能安全治理国际合作，共享最佳实践，提倡建立开放性平台，通过跨学科、跨领域、跨地区、跨国界的对话和合作，推动形成具有广泛共识的全球人工智能治理体系。
第二条 人工智能安全治理框架构成
基于风险管理理念，本框架针对不同类型的人工智能安全风险，从技术、管理两方面提出防范应对措施。同时，目前人工智能研发应用仍在快速发展，安全风险的表现形式、影响程度、认识感知亦随之变化，防范应对措施也将相应动态调整更新，需要各方共同对治理框架持续优化完善。
（一）安全风险方面
通过分析人工智能技术特性，以及在不同行业领域应用场景，梳理人工智能技术本身，及其在应用过程中面临的各种安全风险隐患。
（二）技术应对措施方面
针对模型算法、训练数据、算力设施、产品服务、应用场景，提出通过安全软件开发、数据质量提升、安全建设运维、测评监测加固等技术手段提升人工智能产品及应用的安全性、公平性、可靠性、鲁棒性的措施。
（三）综合治理措施方面
明确技术研发机构、服务提供者、用户、政府部门、行业协会、社会组织等各方发现、防范、应对人工智能安全风险的措施手段，推动各方协同共治。
（四）安全开发应用指引方面
明确模型算法研发者、服务提供者、重点领域用户和社会公众用户，开发应用人工智能技术的若干安全指导规范。
第三条 人工智能安全风险分类（内生风险）
（一）模型算法安全风险
可解释性差的风险
以深度学习为代表的人工智能算法内部运行逻辑复杂，推理过程属黑灰盒模式，可能导致输出结果难以预测和确切归因，如有异常难以快速修正和溯源追责。
偏见、歧视风险
算法设计及训练过程中，个人偏见被有意、无意引入，或者因训练数据集质量问题，导致算法设计目的、输出结果存在偏见或歧视，甚至输出存在民族、宗教、国别、地域等歧视性内容。
鲁棒性弱风险
由于深度神经网络存在非线性、大规模等特点，人工智能易受复杂多变运行环境或恶意干扰、诱导的影响，可能带来性能下降、决策错误等诸多问题。
被窃取、篡改的风险
参数、结构、功能等算法核心信息，面临被逆向攻击窃取、修改，甚至嵌入后门的风险，可导致知识产权被侵犯、商业机密泄露，推理过程不可信、决策输出错误，甚至运行故障。
输出不可靠风险
生成式人工智能可能产生"幻觉"，即生成看似合理，实则不符常理的内容，造成知识偏见与误导。
对抗攻击风险
攻击者通过创建精心设计的对抗样本数据，隐蔽地误导、影响，以至操纵人工智能模型，使其产生错误的输出，甚至造成运行瘫痪。
（二）数据安全风险
违规收集使用数据风险
人工智能训练数据的获取，以及提供服务与用户交互过程中，存在未经同意收集、不当使用数据和个人信息的安全风险。
训练数据含不当内容、被"投毒"风险
训练数据中含有虚假、偏见、侵犯知识产权等违法有害信息，或者来源缺乏多样性，导致输出违法的、不良的、偏激的等有害信息内容。训练数据还面临攻击者篡改、注入错误、误导数据的"投毒"风险，"污染"模型的概率分布，进而造成准确性、可信度下降。
训练数据标注不规范风险
训练数据标注过程中，存在因标注规则不完备、标注人员能力不够、标注错误等问题，不仅会影响模型算法准确度、可靠性、有效性，还可能导致训练偏差、偏见歧视放大、泛化能力不足或输出错误。
数据泄露风险
人工智能研发应用过程中，因数据处理不当、非授权访问、恶意攻击、诱导交互等问题，可能导致数据和个人信息泄露。
（三）系统安全风险
缺陷、后门被攻击利用风险
人工智能算法模型设计、训练和验证的标准接口、特性库和工具包，以及开发界面和执行平台可能存在逻辑缺陷、漏洞等脆弱点，还可能被恶意植入后门，存在被触发和攻击利用的风险。
算力安全风险
人工智能训练运行所依赖的算力基础设施，涉及多源、泛在算力节点，不同类型计算资源，面临算力资源恶意消耗、算力层面风险跨边界传递等风险。
供应链安全风险
人工智能产业链呈现高度全球化分工协作格局。但个别国家利用技术垄断和出口管制等单边强制措施制造发展壁垒，恶意阻断全球人工智能供应链，带来突出的芯片、软件、工具断供风险。
第四条（一） 人工智能应用安全风险
（一）网络域安全风险
信息内容安全风险
人工智能生成或合成内容，易引发虚假信息传播、歧视偏见、隐私泄露、侵权等问题，威胁公民生命财产安全、国家安全、意识形态安全和伦理安全。如果用户输入的提示词存在不良内容，在模型安全防护机制不完善的情况下，有可能输出违法有害内容。
混淆事实、误导用户、绕过鉴权的风险
人工智能系统及输出内容等未经标识，导致用户难以识别交互对象及生成内容来源是否为人工智能系统，难以鉴别生成内容的真实性，影响用户判断，导致误解。同时，人工智能生成图片、音频、视频等高仿真内容，可能绕过现有人脸识别、语音识别等身份认证机制，导致认证鉴权失效。
不当使用引发信息泄露风险
政府、企业等机构工作人员在业务工作中不规范、不当使用人工智能服务，向大模型输入内部业务数据、工业信息，导致工作秘密、商业秘密、敏感业务数据泄露。
滥用于网络攻击的风险
人工智能可被用于实施自动化网络攻击或提高攻击效率，包括挖掘利用漏洞、破解密码、生成恶意代码、发送钓鱼邮件、网络扫描、社会工程学攻击等，降低网络攻击门槛，增大安全防护难度。
模型复用的缺陷传导风险
依托基础模型进行二次开发或微调，是常见的人工智能应用模式，如果基础模型存在安全缺陷，将导致风险传导至下游模型。
（二）现实域安全风险
诱发传统经济社会安全风险
人工智能应用于金融、能源、电信、交通、民生等传统行业领域，如自动驾驶、智能诊疗等，模型算法存在的幻觉输出、错误决策，以及因不当使用、外部攻击等原因出现系统性能下降、中断、失控等问题，将对用户人身生命财产安全、经济社会安全稳定等造成安全威胁。
用于违法犯罪活动的风险
人工智能可能被利用于涉恐、涉暴、涉赌、涉毒等传统违法犯罪活动，包括传授违法犯罪技巧、隐匿违法犯罪行为、制作违法犯罪工具等。
两用物项和技术滥用风险
因不当使用或滥用人工智能两用物项和技术，对国家安全、经济安全、公共卫生安全等带来严重风险。包括极大降低非专家设计、合成、获取、使用核生化导武器的门槛；设计网络武器，通过自动挖掘与利用漏洞等方式，对广泛潜在目标发起网络攻击。
（三）认知域安全风险
加剧"信息茧房"效应风险
人工智能将广泛应用于定制化的信息服务，收集用户信息，分析用户类型、需求、意图、喜好、行为习惯，甚至特定时间段公众主流意识，进而向用户推送程式化、定制化信息及服务，"信息茧房"效应进一步加剧。
用于开展认知战的风险
人工智能可被利用于制作传播虚假新闻、图像、音频、视频等，宣扬恐怖主义、极端主义、有组织犯罪等内容，干涉他国内政、社会制度及社会秩序，危害他国主权；通过社交机器人在网络空间抢占话语权和议程设置权，左右公众价值观和思维认知。
（四）伦理域安全风险
加剧社会歧视偏见、扩大智能鸿沟的风险
利用人工智能收集分析人类行为、社会地位、经济状态、个体性格等，对不同人群进行标识分类、区别对待，带来系统性、结构性的社会歧视与偏见。同时，拉大不同地区人工智能鸿沟。
挑战传统社会秩序的风险
人工智能发展及应用，可能带来生产工具、生产关系的大幅改变，加速重构传统行业模式，颠覆传统的就业观、生育观、教育观，对传统社会秩序的稳定运行带来挑战。
未来脱离控制的风险
随着人工智能技术的快速发展，不排除人工智能自主获取外部资源、自我复制，产生自我意识，寻求外部权力，带来谋求与人类争夺控制权的风险。
第四条（二） 技术应对措施
针对人工智能内生安全风险
模型算法安全风险应对
（1）不断提高人工智能可解释性、可预测性，为人工智能系统内部构造、推理逻辑、技术接口、输出结果提供明确说明，正确反映人工智能系统产生结果的过程。
（2）在设计、研发、部署、维护过程中建立并实施安全开发规范，尽可能消除模型算法存在的安全缺陷、歧视性倾向，提高鲁棒性。
（2）在设计、研发、部署、维护过程中建立并实施安全开发规范，尽可能消除模型算法存在的安全缺陷、歧视性倾向，提高鲁棒性。
数据安全风险应对
（1）在训练数据和用户交互数据的收集、存储、使用、加工、传输、提供、公开、删除等各环节，应遵循数据收集使用、个人信息处理的安全规则，严格落实关于用户控制权、知情权、选择权等法律法规明确的合法权益。
（2）加强知识产权保护，在训练数据选择、结果输出等环节防止侵犯知识产权。
（3）对训练数据进行严格筛选，确保不包含核生化导武器等高危领域敏感数据。
（4）训练数据中如包含敏感个人信息和重要数据，应加强数据安全管理，符合数据安全和个人信息保护相关标准规范。
（5）使用真实、准确、客观、多样且来源合法的训练数据，及时过滤失效、错误、偏见数据。
（6）向境外提供人工智能服务，应符合数据跨境管理规定。向境外提供人工智能模型算法，应符合出口管制要求。
（2）加强知识产权保护，在训练数据选择、结果输出等环节防止侵犯知识产权。
（3）对训练数据进行严格筛选，确保不包含核生化导武器等高危领域敏感数据。
（4）训练数据中如包含敏感个人信息和重要数据，应加强数据安全管理，符合数据安全和个人信息保护相关标准规范。
（5）使用真实、准确、客观、多样且来源合法的训练数据，及时过滤失效、错误、偏见数据。
（6）向境外提供人工智能服务，应符合数据跨境管理规定。向境外提供人工智能模型算法，应符合出口管制要求。
系统安全风险应对
（1）对人工智能技术和产品的原理、能力、适用场景、安全风险适当公开，对输出内容进行明晰标识，不断提高人工智能系统透明性。
（2）对聚合多个人工智能模型或系统的平台，应加强风险识别、检测、防护，防止因平台恶意行为或被攻击入侵影响承载的人工智能模型或系统。
（3）加强人工智能算力平台和系统服务的安全建设、管理、运维能力，确保基础设施和服务运行不中断。
（4）对于人工智能系统采用的芯片、软件、工具、算力和数据资源，应高度关注供应链安全。跟踪软硬件产品的漏洞、缺陷信息并及时采取修补加固措施，保证系统安全性。
（2）对聚合多个人工智能模型或系统的平台，应加强风险识别、检测、防护，防止因平台恶意行为或被攻击入侵影响承载的人工智能模型或系统。
（3）加强人工智能算力平台和系统服务的安全建设、管理、运维能力，确保基础设施和服务运行不中断。
（4）对于人工智能系统采用的芯片、软件、工具、算力和数据资源，应高度关注供应链安全。跟踪软硬件产品的漏洞、缺陷信息并及时采取修补加固措施，保证系统安全性。
针对人工智能应用安全风险
网络域 / 现实域 / 认知域 / 伦理域风险应对
（1）建立安全防护机制，防止模型运行过程中被干扰、篡改而输出不可信结果。
（2）应建立数据护栏，确保人工智能系统输出敏感个人信息和重要数据符合相关法律法规。
（3）根据用户实际应用场景设置服务提供边界，裁减人工智能系统可能被滥用的功能，系统提供服务时不应超出预设应用范围。
（4）提高人工智能系统最终用途追溯能力，防止被用于核生化导等大规模杀伤性武器制造等高危场景。
（5）通过技术手段判别不符合预期、不真实、不准确的输出结果，并依法依规监管。
（6）对收集用户提问信息进行关联分析、汇聚挖掘，进而判断用户身份、喜好以及个人思想倾向的人工智能系统，应严格防范其滥用。
（7）加强对人工智能生成合成内容的检测技术研发，提升对认知战手段的防范、检测、处置能力。
（8）在算法设计、模型训练和优化、提供服务等过程中，应采取训练数据筛选、输出校验等方式，防止产生民族、信仰、国别、地域、性别、年龄、职业、健康等方面歧视。
（9）应用于政府部门、关键信息基础设施以及直接影响公共安全和公民生命健康安全的领域等重点领域的人工智能系统，应具备高效精准的应急管控措施。
（2）应建立数据护栏，确保人工智能系统输出敏感个人信息和重要数据符合相关法律法规。
（3）根据用户实际应用场景设置服务提供边界，裁减人工智能系统可能被滥用的功能，系统提供服务时不应超出预设应用范围。
（4）提高人工智能系统最终用途追溯能力，防止被用于核生化导等大规模杀伤性武器制造等高危场景。
（5）通过技术手段判别不符合预期、不真实、不准确的输出结果，并依法依规监管。
（6）对收集用户提问信息进行关联分析、汇聚挖掘，进而判断用户身份、喜好以及个人思想倾向的人工智能系统，应严格防范其滥用。
（7）加强对人工智能生成合成内容的检测技术研发，提升对认知战手段的防范、检测、处置能力。
（8）在算法设计、模型训练和优化、提供服务等过程中，应采取训练数据筛选、输出校验等方式，防止产生民族、信仰、国别、地域、性别、年龄、职业、健康等方面歧视。
（9）应用于政府部门、关键信息基础设施以及直接影响公共安全和公民生命健康安全的领域等重点领域的人工智能系统，应具备高效精准的应急管控措施。
第五条 综合治理措施
（一）实施人工智能应用分类分级管理
根据功能、性能、应用场景等，对人工智能系统分类分级，建立风险等级测试评估体系。加强人工智能最终用途管理，对特定人群及场景下使用人工智能技术提出相关要求，防止人工智能系统被滥用。对算力、推理能力达到一定阈值或应用在特定行业领域的人工智能系统进行登记备案，要求其具备在设计、研发、测试、部署、使用、维护等全生命周期的安全防护能力。
（二）建立人工智能服务可追溯管理制度
对面向公众服务的人工智能系统，通过数字证书技术对其进行标识管理。制定出台人工智能生成合成内容标识标准规范，明确显式、隐式等标识要求，全面覆盖制作源头、传播路径、分发渠道等关键环节，便于用户识别判断信息来源及真实性。
（三）完善人工智能数据安全和个人信息保护规范
针对人工智能技术及应用特点，明确人工智能训练、标注、使用、输出等各环节的数据安全和个人信息保护要求。
（四）构建负责任的人工智能研发应用体系
研究提出"以人为本、智能向善"在人工智能研发应用中的具体操作指南和最佳实践，持续推进人工智能设计、研发、应用的价值观、伦理观对齐。探索适应人工智能时代的版权保护和开发利用制度，持续推进高质量基础语料库和数据集建设，为人工智能安全发展提供优质营养供给。制定人工智能伦理审查准则、规范和指南，完善伦理审查制度。
（五）强化人工智能供应链安全保障
推动共享人工智能知识成果，开源人工智能技术，共同研发人工智能芯片、框架、软件，引导产业界建立开放生态，增强供应链来源多样性，保障人工智能供应链安全性稳定性。
（六）推进人工智能可解释性研究
从机器学习理论、训练方法、人机交互等方面组织研究人工智能决策透明度、可信度、纠错机制等问题，不断提高人工智能可解释性和可预测性，避免人工智能系统意外决策产生恶意行为。
（七）人工智能安全风险威胁信息共享和应急处置机制
持续跟踪分析人工智能技术、软硬件产品、服务等方面存在的安全漏洞、缺陷、风险威胁、安全事件等动向，协调有关研发者、服务提供者建立风险威胁信息通报和共享机制。构建人工智能安全事件应急处置机制，制定应急预案，开展应急演练，及时快速有效处置人工智能安全威胁和事件。
（八）加大人工智能安全人才培养力度
推动人工智能安全教育与人工智能学科同步发展，依托学校、科研机构等加强人工智能安全设计、开发、治理人才的培养，支持培养人工智能安全前沿基础领域顶尖人才，壮大无人驾驶、智能医疗、类脑智能、脑机接口等领域安全人才队伍。
（九）建立健全人工智能安全宣传教育、行业自律、社会监督机制
面向政府、企业、社会公用事业单位加强人工智能安全规范应用的教育培训。加强人工智能安全风险及防范应对知识的宣传，全面提高全社会人工智能安全意识。指导支持网络安全、人工智能领域行业协会加强行业自律，制定提出高于监管要求、具有引领示范作用的人工智能安全自律公约，引导督促人工智能技术研发机构、服务提供者持续提升安全能力水平；面向公众建立人工智能安全风险隐患投诉举报受理机制，形成有效的人工智能安全社会监督氛围。
（十）促进人工智能安全治理国际交流合作
积极与各国就人工智能开展合作交流，支持在联合国框架下成立国际人工智能治理机构，协调人工智能发展、安全与治理重大问题。推进APEC、G20、金砖国家等多边机制下的人工智能安全治理合作，加强与共建"一带一路"国家、"全球南方"国家合作，研究成立人工智能安全治理联盟，增强发展中国家在全球人工智能治理中的代表性和发言权。鼓励人工智能企业、机构开展跨国交流合作，分享最佳操作实践，共同制定人工智能安全国际标准。
第六条 人工智能安全开发应用指引
（一）模型算法研发者安全开发指引
研发者与服务提供者指引（10条）
（1）研发者应在需求分析、项目立项、模型设计开发、训练数据选用等关键环节，切实践行"以人为本、智能向善"理念宗旨，遵循科技伦理规范，采取开展内部研讨、组织专家评议、科技伦理审查、听取公众意见、与潜在目标用户沟通交流、加强员工安全教育培训等措施。
（2）服务提供者应在合同或服务协议中，以使用者易于理解的方式，告知人工智能产品和服务的适用范围、注意事项、使用禁忌，支持使用者知情选择、审慎使用。
（3）服务提供者应在告知同意、服务协议等文件中，支持使用者行使人类监督和控制责任。
（4）服务提供者应让使用者了解人工智能产品的精确度，在人工智能决策有重大影响时，做好解释说明预案。
（5）服务提供者应检查研发者提供的责任说明文件，确保责任链条可以追溯到递归采用的人工智能模型。
（6）服务提供者应提高人工智能风险防范意识，建立健全实时风险监控管理机制，持续跟踪运行中安全风险。
（7）服务提供者应评估人工智能产品与服务在面临故障、攻击等异常条件下抵御或克服不利条件的能力，防范出现意外结果和行为错误，确保最低限度有效功能。
（8）服务提供者应将人工智能系统运行中发现的安全事故、安全漏洞等及时向主管部门报告。
（9）服务提供者应在合同或服务协议中明确，一旦发现不符合使用意图和说明限制的误用、滥用，服务提供者有权采取纠正措施或提前终止服务。
（10）服务提供者应评估人工智能产品对使用者的影响，防止对使用者身心健康、生命财产等造成危害。
（2）服务提供者应在合同或服务协议中，以使用者易于理解的方式，告知人工智能产品和服务的适用范围、注意事项、使用禁忌，支持使用者知情选择、审慎使用。
（3）服务提供者应在告知同意、服务协议等文件中，支持使用者行使人类监督和控制责任。
（4）服务提供者应让使用者了解人工智能产品的精确度，在人工智能决策有重大影响时，做好解释说明预案。
（5）服务提供者应检查研发者提供的责任说明文件，确保责任链条可以追溯到递归采用的人工智能模型。
（6）服务提供者应提高人工智能风险防范意识，建立健全实时风险监控管理机制，持续跟踪运行中安全风险。
（7）服务提供者应评估人工智能产品与服务在面临故障、攻击等异常条件下抵御或克服不利条件的能力，防范出现意外结果和行为错误，确保最低限度有效功能。
（8）服务提供者应将人工智能系统运行中发现的安全事故、安全漏洞等及时向主管部门报告。
（9）服务提供者应在合同或服务协议中明确，一旦发现不符合使用意图和说明限制的误用、滥用，服务提供者有权采取纠正措施或提前终止服务。
（10）服务提供者应评估人工智能产品对使用者的影响，防止对使用者身心健康、生命财产等造成危害。
（二）重点领域使用者安全应用指引
重点领域使用者指引（9条）
（1）对于政府部门、关键信息基础设施以及直接影响公共安全和公民生命健康安全的领域等重点领域使用者，应审慎评估目标应用场景采用人工智能技术后带来的长期和潜在影响，开展风险评估与定级，避免技术滥用。
（2）重点领域使用者应根据人工智能系统的适用场景、安全性、可靠性、可控性等，定期进行系统审计，加强风险防范意识与风险应对处置能力。
（3）重点领域使用者在使用人工智能产品前，应全面了解其数据处理和隐私保护措施。
（4）重点领域使用者应使用高安全级别的密码策略，启用多因素认证机制，增强账户安全性。
（5）重点领域使用者应增强网络安全、供应链安全等方面的能力，降低人工智能系统被攻击、重要数据被窃取或泄露的风险，保障业务不中断。
（6）重点领域使用者应合理限制人工智能系统对数据的访问权限，制定数据备份和恢复计划，定期对数据处理流程进行检查。
（7）重点领域使用者应确保操作符合保密规定，在处理敏感数据时使用加密技术等保护措施。
（8）重点领域使用者应对人工智能行为和影响进行有效监督，确保人工智能产品和服务的运行基于人的授权、处于人的控制之下。
（9）重点领域使用者应避免完全依赖人工智能的决策，监控及记录未采纳人工智能决策的情况，并对决策不一致进行分析，在遭遇事故时具备及时切换到人工或传统系统等的能力。
（2）重点领域使用者应根据人工智能系统的适用场景、安全性、可靠性、可控性等，定期进行系统审计，加强风险防范意识与风险应对处置能力。
（3）重点领域使用者在使用人工智能产品前，应全面了解其数据处理和隐私保护措施。
（4）重点领域使用者应使用高安全级别的密码策略，启用多因素认证机制，增强账户安全性。
（5）重点领域使用者应增强网络安全、供应链安全等方面的能力，降低人工智能系统被攻击、重要数据被窃取或泄露的风险，保障业务不中断。
（6）重点领域使用者应合理限制人工智能系统对数据的访问权限，制定数据备份和恢复计划，定期对数据处理流程进行检查。
（7）重点领域使用者应确保操作符合保密规定，在处理敏感数据时使用加密技术等保护措施。
（8）重点领域使用者应对人工智能行为和影响进行有效监督，确保人工智能产品和服务的运行基于人的授权、处于人的控制之下。
（9）重点领域使用者应避免完全依赖人工智能的决策，监控及记录未采纳人工智能决策的情况，并对决策不一致进行分析，在遭遇事故时具备及时切换到人工或传统系统等的能力。
（三）社会公众安全应用指引
社会公众指引（6条）
（1）社会公众应提高对人工智能产品安全风险的认识，选择信誉良好的人工智能产品。
（2）社会公众应在使用前仔细阅读产品合同或服务协议，了解产品的功能、限制和隐私政策，准确认知人工智能产品做出判断决策的局限性，合理设定使用预期。
（3）社会公众应提高个人信息保护意识，避免在不必要的情况下输入敏感信息。
（4）社会公众应了解人工智能产品的数据处理方式，避免使用不符合隐私保护原则的产品。
（5）社会公众在使用人工智能产品时，应关注网络安全风险，避免人工智能产品成为网络攻击的目标。
（6）社会公众应注意人工智能产品对儿童和青少年的影响，预防沉迷及过度使用。
（2）社会公众应在使用前仔细阅读产品合同或服务协议，了解产品的功能、限制和隐私政策，准确认知人工智能产品做出判断决策的局限性，合理设定使用预期。
（3）社会公众应提高个人信息保护意识，避免在不必要的情况下输入敏感信息。
（4）社会公众应了解人工智能产品的数据处理方式，避免使用不符合隐私保护原则的产品。
（5）社会公众在使用人工智能产品时，应关注网络安全风险，避免人工智能产品成为网络攻击的目标。
（6）社会公众应注意人工智能产品对儿童和青少年的影响，预防沉迷及过度使用。
