> $GITHUB_OUTPUT
+ echo "$SPELLCHECK_LOG" >> $GITHUB_OUTPUT
+ echo "SPELLEOF" >> $GITHUB_OUTPUT
+
+ if [ $FINAL_EXIT_CODE -ne 0 ]; then
+ echo "Spell check failed! See above for details."
+ echo
+ echo "Here are a few tips:"
+ echo "- All PyTorch API objects must be in double backticks or use an intersphinx directive."
+ echo " Example: ``torch.nn``, :func:"
+ echo "- Consult en-wordlist.txt for spellings of some of the words."
+ echo " You can add a word to en-wordlist.txt if:"
+ echo " 1) It's a common abbreviation, like RNN."
+ echo " 2) It's a word widely accepted in the industry."
+ echo "- Please do not add words like 'dtype', 'torch.nn.Transformer' to pass spellcheck."
+ echo " Instead wrap it in double backticks or use an intersphinx directive."
+ echo
+ exit 1
+ fi
diff --git a/.gitignore b/.gitignore
index 1d9d572e565..3f1f927ee33 100644
--- a/.gitignore
+++ b/.gitignore
@@ -5,6 +5,8 @@ advanced
pytorch_basics
/recipes
prototype
+/unstable
+sg_execution_times.rst
#data things
_data/
@@ -127,3 +129,6 @@ cleanup.sh
# pyspelling
dictionary.dic
+
+# linters
+/.lintbin
diff --git a/.gitmodules b/.gitmodules
index 3a3c564c8fa..e69de29bb2d 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -1,3 +0,0 @@
-[submodule "src/pytorch-sphinx-theme"]
- path = src/pytorch-sphinx-theme
- url = https://github.com/pytorch/pytorch_sphinx_theme
diff --git a/.jenkins/build.sh b/.jenkins/build.sh
index cf3da8461e5..32ceec660da 100755
--- a/.jenkins/build.sh
+++ b/.jenkins/build.sh
@@ -19,15 +19,21 @@ sudo apt-get install -y pandoc
# NS: Path to python runtime should already be part of docker container
# export PATH=/opt/conda/bin:$PATH
-#Install PyTorch Nightly for test.
-# Nightly - pip install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu102/torch_nightly.html
-# Install 2.2 for testing - uncomment to install nightly binaries (update the version as needed).
-# pip uninstall -y torch torchvision torchaudio torchtext torchdata
-# pip3 install torch==2.3.0 torchvision torchaudio --no-cache-dir --index-url https://download.pytorch.org/whl/test/cu121
+# Install PyTorch Nightly for test.
+if [ "${USE_NIGHTLY:-0}" -eq 1 ]; then
+ sudo pip uninstall -y torch torchvision torchaudio
+ pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130
+ pip show torch
+fi
+# Nightly - pip install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu102/torch_nightly.html
+# Install 2.5 to merge all 2.4 PRs - uncomment to install nightly binaries (update the version as needed).
+# sudo pip uninstall -y fbgemm-gpu torchrec
+# sudo pip uninstall -y torch torchvision torchaudio torchtext torchdata torchrl tensordict
+# sudo pip3 install fbgemm-gpu==1.1.0 torchrec==1.0.0 --no-cache-dir --index-url https://download.pytorch.org/whl/test/cu124
+# pip3 install torch==2.7.0 torchvision torchaudio --no-cache-dir --index-url https://download.pytorch.org/whl/test/cu126
# Install two language tokenizers for Translation with TorchText tutorial
-python -m spacy download en_core_web_sm
-python -m spacy download de_core_news_sm
+pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.0/en_core_web_sm-3.4.0-py3-none-any.whl
awsv2 -i
awsv2 configure set default.s3.multipart_threshold 5120MB
@@ -114,8 +120,10 @@ if [[ "${JOB_TYPE}" == "worker" ]]; then
python .jenkins/validate_tutorials_built.py
# Step 6: Copy generated files to S3, tag with commit ID
- 7z a worker_${WORKER_ID}.7z docs
- awsv2 s3 cp worker_${WORKER_ID}.7z s3://${BUCKET_NAME}/${COMMIT_ID}/worker_${WORKER_ID}.7z
+ if [ "${UPLOAD:-0}" -eq 1 ]; then
+ 7z a worker_${WORKER_ID}.7z docs
+ awsv2 s3 cp worker_${WORKER_ID}.7z s3://${BUCKET_NAME}/${COMMIT_ID}/worker_${WORKER_ID}.7z
+ fi
elif [[ "${JOB_TYPE}" == "manager" ]]; then
# Step 1: Generate no-plot HTML pages for all tutorials
pip3 install -e git+https://github.com/pytorch/pytorch_sphinx_theme.git#egg=pytorch_sphinx_theme
@@ -150,6 +158,12 @@ elif [[ "${JOB_TYPE}" == "manager" ]]; then
# Step 7: push new HTML files and static files to gh-pages
if [[ "$COMMIT_SOURCE" == "refs/heads/master" || "$COMMIT_SOURCE" == "refs/heads/main" ]]; then
git clone https://github.com/pytorch/tutorials.git -b gh-pages gh-pages
+ # Clean up directories that contain tutorials
+
+ for dir in beginner intermediate prototype recipes advanced distributed vision text audio; do
+ rm -rf "gh-pages/$dir"
+ done
+
cp -r docs/* gh-pages/
pushd gh-pages
# DANGER! DO NOT REMOVE THE `set +x` SETTING HERE!
diff --git a/.jenkins/download_data.py b/.jenkins/download_data.py
index cc07c72561b..939e63fc7a8 100644
--- a/.jenkins/download_data.py
+++ b/.jenkins/download_data.py
@@ -12,7 +12,7 @@
BEGINNER_DATA_DIR = REPO_BASE_DIR / "beginner_source" / "data"
INTERMEDIATE_DATA_DIR = REPO_BASE_DIR / "intermediate_source" / "data"
ADVANCED_DATA_DIR = REPO_BASE_DIR / "advanced_source" / "data"
-PROTOTYPE_DATA_DIR = REPO_BASE_DIR / "prototype_source" / "data"
+PROTOTYPE_DATA_DIR = REPO_BASE_DIR / "unstable_source" / "data"
FILES_TO_RUN = os.getenv("FILES_TO_RUN")
@@ -106,7 +106,7 @@ def download_lenet_mnist() -> None:
)
def download_gpu_quantization_torchao() -> None:
- # Download SAM model checkpoint for prototype_source/gpu_quantization_torchao_tutorial.py
+ # Download SAM model checkpoint unstable_source/gpu_quantization_torchao_tutorial.py
download_url_to_file("https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth",
prefix=PROTOTYPE_DATA_DIR,
dst="sam_vit_h_4b8939.pth",
diff --git a/.jenkins/insert_last_verified.py b/.jenkins/insert_last_verified.py
new file mode 100644
index 00000000000..b43ef8de8e8
--- /dev/null
+++ b/.jenkins/insert_last_verified.py
@@ -0,0 +1,160 @@
+import json
+import os
+import subprocess
+import sys
+from datetime import datetime
+
+from bs4 import BeautifulSoup
+
+
+json_file_path = "tutorials-review-data.json"
+
+# paths to skip from the post-processing script
+paths_to_skip = [
+ "beginner/examples_autograd/two_layer_net_custom_function", # not present in the repo
+ "beginner/examples_nn/two_layer_net_module", # not present in the repo
+ "beginner/examples_tensor/two_layer_net_numpy", # not present in the repo
+ "beginner/examples_tensor/two_layer_net_tensor", # not present in the repo
+ "beginner/examples_autograd/two_layer_net_autograd", # not present in the repo
+ "beginner/examples_nn/two_layer_net_optim", # not present in the repo
+ "beginner/examples_nn/two_layer_net_nn", # not present in the repo
+ "intermediate/coding_ddpg", # not present in the repo - will delete the carryover
+]
+# Mapping of source directories to build directories
+source_to_build_mapping = {
+ "beginner": "beginner_source",
+ "recipes": "recipes_source",
+ "distributed": "distributed",
+ "intermediate": "intermediate_source",
+ "prototype": "prototype_source",
+ "advanced": "advanced_source",
+ "": "", # root dir for index.rst
+}
+
+def get_git_log_date(file_path, git_log_args):
+ try:
+ result = subprocess.run(
+ ["git", "log"] + git_log_args + ["--", file_path],
+ capture_output=True,
+ text=True,
+ check=True,
+ )
+ if result.stdout:
+ date_str = result.stdout.splitlines()[0]
+ return datetime.strptime(date_str, "%a, %d %b %Y %H:%M:%S %z")
+ except subprocess.CalledProcessError:
+ pass
+ raise ValueError(f"Could not find date for {file_path}")
+
+def get_creation_date(file_path):
+ return get_git_log_date(file_path, ["--diff-filter=A", "--format=%aD"]).strftime("%b %d, %Y")
+
+
+def get_last_updated_date(file_path):
+ return get_git_log_date(file_path, ["-1", "--format=%aD"]).strftime("%b %d, %Y")
+
+# Try to find the source file with the given base path and the extensions .rst and .py
+def find_source_file(base_path):
+ for ext in [".rst", ".py"]:
+ source_file_path = base_path + ext
+ if os.path.exists(source_file_path):
+ return source_file_path
+ return None
+
+
+# Function to process a JSON file and insert the "Last Verified" information into the HTML files
+def process_json_file(build_dir , json_file_path):
+ with open(json_file_path, "r", encoding="utf-8") as json_file:
+ json_data = json.load(json_file)
+
+ for entry in json_data:
+ path = entry["Path"]
+ last_verified = entry["Last Verified"]
+ status = entry.get("Status", "")
+ if path in paths_to_skip:
+ print(f"Skipping path: {path}")
+ continue
+ if status in ["needs update", "not verified"]:
+ formatted_last_verified = "Not Verified"
+ elif last_verified:
+ try:
+ last_verified_date = datetime.strptime(last_verified, "%Y-%m-%d")
+ formatted_last_verified = last_verified_date.strftime("%b %d, %Y")
+ except ValueError:
+ formatted_last_verified = "Unknown"
+ else:
+ formatted_last_verified = "Not Verified"
+ if status == "deprecated":
+ formatted_last_verified += "Deprecated"
+
+ for build_subdir, source_subdir in source_to_build_mapping.items():
+ if path.startswith(build_subdir):
+ html_file_path = os.path.join(build_dir, path + ".html")
+ base_source_path = os.path.join(
+ source_subdir, path[len(build_subdir) + 1 :]
+ )
+ source_file_path = find_source_file(base_source_path)
+ break
+ else:
+ print(f"Warning: No mapping found for path {path}")
+ continue
+
+ if not os.path.exists(html_file_path):
+ print(
+ f"Warning: HTML file not found for path {html_file_path}."
+ "If this is a new tutorial, please add it to the audit JSON file and set the Verified status and todays's date."
+ )
+ continue
+
+ if not source_file_path:
+ print(f"Warning: Source file not found for path {base_source_path}.")
+ continue
+
+ created_on = get_creation_date(source_file_path)
+ last_updated = get_last_updated_date(source_file_path)
+
+ with open(html_file_path, "r", encoding="utf-8") as file:
+ soup = BeautifulSoup(file, "html.parser")
+ # Check if the tag with class "date-info-last-verified" already exists
+ existing_date_info = soup.find("p", {"class": "date-info-last-verified"})
+ if existing_date_info:
+ print(
+ f"Warning:
tag with class 'date-info-last-verified' already exists in {html_file_path}"
+ )
+ continue
+
+ h1_tag = soup.find("h1") # Find the h1 tag to insert the dates
+ if h1_tag:
+ date_info_tag = soup.new_tag("p", **{"class": "date-info-last-verified"})
+ date_info_tag["style"] = "color: #6c6c6d; font-size: small;"
+ # Add the "Created On", "Last Updated", and "Last Verified" information
+ date_info_tag.string = (
+ f"Created On: {created_on} | "
+ f"Last Updated: {last_updated} | "
+ f"Last Verified: {formatted_last_verified}"
+ )
+ # Insert the new tag after the
tag
+ h1_tag.insert_after(date_info_tag)
+ # Save back to the HTML.
+ with open(html_file_path, "w", encoding="utf-8") as file:
+ file.write(str(soup))
+ else:
+ print(f"Warning: tag not found in {html_file_path}")
+
+
+def main():
+ if len(sys.argv) < 2:
+ print("Error: Build directory not provided. Exiting.")
+ exit(1)
+ build_dir = sys.argv[1]
+ print(f"Build directory: {build_dir}")
+ process_json_file(build_dir , json_file_path)
+ print(
+ "Finished processing JSON file. Please check the output for any warnings. "
+ "Pages like `nlp/index.html` are generated only during the full `make docs` "
+ "or `make html` build. Warnings about these files when you run `make html-noplot` "
+ "can be ignored."
+ )
+
+if __name__ == "__main__":
+ main()
diff --git a/.jenkins/metadata.json b/.jenkins/metadata.json
index 4814f9a7d2b..6e82d054b4e 100644
--- a/.jenkins/metadata.json
+++ b/.jenkins/metadata.json
@@ -28,6 +28,12 @@
"intermediate_source/model_parallel_tutorial.py": {
"needs": "linux.16xlarge.nvidia.gpu"
},
+ "intermediate_source/torchrec_intro_tutorial.py": {
+ "needs": "linux.g5.4xlarge.nvidia.gpu"
+ },
+ "recipes_source/torch_export_aoti_python.py": {
+ "needs": "linux.g5.4xlarge.nvidia.gpu"
+ },
"advanced_source/pendulum.py": {
"needs": "linux.g5.4xlarge.nvidia.gpu",
"_comment": "need to be here for the compiling_optimizer_lr_scheduler.py to run."
@@ -52,9 +58,15 @@
"intermediate_source/scaled_dot_product_attention_tutorial.py": {
"needs": "linux.g5.4xlarge.nvidia.gpu"
},
+ "intermediate_source/transformer_building_blocks.py": {
+ "needs": "linux.g5.4xlarge.nvidia.gpu"
+ },
"recipes_source/torch_compile_user_defined_triton_kernel_tutorial.py": {
"needs": "linux.g5.4xlarge.nvidia.gpu"
},
+ "recipes_source/regional_compilation.py": {
+ "needs": "linux.g5.4xlarge.nvidia.gpu"
+ },
"advanced_source/semi_structured_sparse.py": {
"needs": "linux.g5.4xlarge.nvidia.gpu"
},
diff --git a/.jenkins/post_process_notebooks.py b/.jenkins/post_process_notebooks.py
index 81f51766c3e..d10eb5a1bcc 100644
--- a/.jenkins/post_process_notebooks.py
+++ b/.jenkins/post_process_notebooks.py
@@ -5,14 +5,14 @@
"""
This post-processing script needs to run after the .ipynb files are
generated. The script removes extraneous ```{=html} syntax from the
-admonitions and splits the cells that have video iframe into a
+admonitions and splits the cells that have video iframe into a
separate code cell that can be run to load the video directly
in the notebook. This script is included in build.sh.
"""
# Pattern to search ``` {.python .jupyter-code-cell}
-pattern = re.compile(r'(.*?)``` {.python .jupyter-code-cell}\n\n(from IPython.display import display, HTML\nhtml_code = """\n.*?\n"""\ndisplay\(HTML\(html_code\)\))\n```(.*)', re.DOTALL)
+pattern = re.compile(r'(.*?)``` {\.python \.jupyter-code-cell}\n(.*?from IPython\.display import display, HTML.*?display\(HTML\(html_code\)\))\n```(.*)', re.DOTALL)
def process_video_cell(notebook_path):
@@ -36,7 +36,7 @@ def process_video_cell(notebook_path):
before_html_block = match.group(1)
code_block = match.group(2)
- # Add a comment to run the cell to display the video
+ # Add a comment to run the cell to display the video
code_block = "# Run this cell to load the video\n" + code_block
# Create a new code cell
new_code_cell = nbf.v4.new_code_cell(source=code_block)
diff --git a/.jenkins/validate_tutorials_built.py b/.jenkins/validate_tutorials_built.py
index 03101df481c..75dd51dd789 100644
--- a/.jenkins/validate_tutorials_built.py
+++ b/.jenkins/validate_tutorials_built.py
@@ -10,40 +10,26 @@
NOT_RUN = [
"beginner_source/basics/intro", # no code
+ "beginner_source/introyt/introyt_index", # no code
"beginner_source/onnx/intro_onnx",
- "beginner_source/translation_transformer",
"beginner_source/profiler",
"beginner_source/saving_loading_models",
"beginner_source/introyt/captumyt",
"beginner_source/examples_nn/polynomial_module",
"beginner_source/examples_nn/dynamic_net",
"beginner_source/examples_nn/polynomial_optim",
- "beginner_source/former_torchies/autograd_tutorial_old",
- "beginner_source/former_torchies/tensor_tutorial_old",
"beginner_source/examples_autograd/polynomial_autograd",
"beginner_source/examples_autograd/polynomial_custom_function",
- "beginner_source/torchtext_custom_dataset_tutorial", # not building with 2.3 RC, might be able to turn on with GA
- "beginner_source/text_sentiment_ngrams_tutorial", # not building with 2.3 RC, might be able to turn on with GA
- "beginner_source/t5_tutorial", # re-enable after this is fixed: https://github.com/pytorch/text/issues/1756
+ "intermediate_source/dqn_with_rnn_tutorial", #not working on 2.8 release reenable after 3514
"intermediate_source/mnist_train_nas", # used by ax_multiobjective_nas_tutorial.py
- "intermediate_source/fx_conv_bn_fuser",
+ "intermediate_source/torch_compile_conv_bn_fuser",
"intermediate_source/_torch_export_nightly_tutorial", # does not work on release
- "advanced_source/super_resolution_with_onnxruntime",
- "advanced_source/python_custom_ops", # https://github.com/pytorch/pytorch/issues/127443
"advanced_source/usb_semisup_learn", # fails with CUDA OOM error, should try on a different worker
- "prototype_source/fx_graph_mode_ptq_dynamic",
- "prototype_source/vmap_recipe",
- "prototype_source/torchscript_freezing",
- "prototype_source/nestedtensor",
- "recipes_source/recipes/saving_and_loading_models_for_inference",
- "recipes_source/recipes/saving_multiple_models_in_one_file",
+ "unstable_source/gpu_direct_storage", # requires specific filesystem + GPUDirect Storage to be set up
"recipes_source/recipes/tensorboard_with_pytorch",
"recipes_source/recipes/what_is_state_dict",
"recipes_source/recipes/profiler_recipe",
- "recipes_source/recipes/save_load_across_devices",
"recipes_source/recipes/warmstarting_model_using_parameters_from_a_different_model",
- "recipes_source/recipes/dynamic_quantization",
- "recipes_source/recipes/saving_and_loading_a_general_checkpoint",
"recipes_source/recipes/benchmark",
"recipes_source/recipes/tuning_guide",
"recipes_source/recipes/zeroing_out_gradients",
@@ -51,12 +37,9 @@
"recipes_source/recipes/timer_quick_start",
"recipes_source/recipes/amp_recipe",
"recipes_source/recipes/Captum_Recipe",
- "intermediate_source/flask_rest_api_tutorial",
- "intermediate_source/text_to_speech_with_torchaudio",
"intermediate_source/tensorboard_profiler_tutorial", # reenable after 2.0 release.
- "intermediate_source/inductor_debug_cpu", # reenable after 2942
- "beginner_source/onnx/onnx_registry_tutorial", # reenable after 2941 is fixed.
- "intermediate_source/torch_export_tutorial" # reenable after 2940 is fixed.
+ "advanced_source/semi_structured_sparse", # reenable after 3303 is fixed.
+ "intermediate_source/torchrec_intro_tutorial.py", #failing with 2.8 reenable after 3498
]
def tutorial_source_dirs() -> List[Path]:
diff --git a/.lintrunner.toml b/.lintrunner.toml
new file mode 100644
index 00000000000..d3a1cbd9885
--- /dev/null
+++ b/.lintrunner.toml
@@ -0,0 +1,225 @@
+merge_base_with = "origin/main"
+
+# 4805a6ead6f1e7f32351056e2602be4e908f69b7 is from pytorch/pytorch main branch 2025-07-16
+
+[[linter]]
+code = 'SPACES'
+include_patterns = ['**']
+exclude_patterns = [
+ "_static/**/*", # Contains some files that should usually not be linted
+ # All files below this should be checked and either removed from the
+ # exclusion list by fixing them or have a reason to be excluded.
+ "advanced_source/coding_ddpg.py",
+ "advanced_source/cpp_autograd.rst",
+ "advanced_source/cpp_custom_ops.rst",
+ "advanced_source/generic_join.rst",
+ "advanced_source/neural_style_tutorial.py",
+ "advanced_source/pendulum.py",
+ "advanced_source/privateuseone.rst",
+ "advanced_source/semi_structured_sparse.py",
+ "advanced_source/sharding.rst",
+ "advanced_source/torch_script_custom_classes/custom_class_project/custom_test.py",
+ "advanced_source/transformer__timeseries_cpp_tutorial/transformer_timeseries.cpp",
+ "advanced_source/usb_semisup_learn.py",
+ "beginner_source/blitz/README.txt",
+ "beginner_source/blitz/neural_networks_tutorial.py",
+ "beginner_source/dcgan_faces_tutorial.py",
+ "beginner_source/ddp_series_fault_tolerance.rst",
+ "beginner_source/ddp_series_theory.rst",
+ "beginner_source/examples_nn/polynomial_module.py",
+ "beginner_source/examples_nn/polynomial_nn.py",
+ "beginner_source/hta_intro_tutorial.rst",
+ "beginner_source/hta_trace_diff_tutorial.rst",
+ "beginner_source/hybrid_frontend/README.txt",
+ "beginner_source/hybrid_frontend_tutorial.rst",
+ "beginner_source/hyperparameter_tuning_tutorial.py",
+ "beginner_source/introyt/README.txt",
+ "beginner_source/introyt/autogradyt_tutorial.py",
+ "beginner_source/introyt/captumyt.py",
+ "beginner_source/introyt/introyt1_tutorial.py",
+ "beginner_source/introyt/modelsyt_tutorial.py",
+ "beginner_source/introyt/tensorboardyt_tutorial.py",
+ "beginner_source/introyt/tensors_deeper_tutorial.py",
+ "beginner_source/introyt/trainingyt.py",
+ "beginner_source/knowledge_distillation_tutorial.py",
+ "beginner_source/nlp/sequence_models_tutorial.py",
+ "beginner_source/onnx/export_control_flow_model_to_onnx_tutorial.py",
+ "beginner_source/onnx/onnx_registry_tutorial.py",
+ "beginner_source/pytorch_with_examples.rst",
+ "beginner_source/saving_loading_models.py",
+ "beginner_source/template_tutorial.py",
+ "beginner_source/transfer_learning_tutorial.py",
+ "intermediate_source/TCPStore_libuv_backend.rst",
+ "intermediate_source/ax_multiobjective_nas_tutorial.py",
+ "intermediate_source/compiled_autograd_tutorial.rst",
+ "intermediate_source/ddp_series_multinode.rst",
+ "intermediate_source/dqn_with_rnn_tutorial.py",
+ "intermediate_source/fx_profiling_tutorial.py",
+ "intermediate_source/inductor_debug_cpu.py",
+ "intermediate_source/jacobians_hessians.py",
+ "intermediate_source/optimizer_step_in_backward_tutorial.py",
+ "intermediate_source/per_sample_grads.py",
+ "intermediate_source/pruning_tutorial.py",
+ "intermediate_source/reinforcement_q_learning.py",
+ "intermediate_source/tensorboard_profiler_tutorial.py",
+ "intermediate_source/torch_compile_tutorial.py",
+ "intermediate_source/transformer_building_blocks.py",
+ "unstable_source/README.md",
+ "unstable_source/README.txt",
+ "unstable_source/gpu_direct_storage.py",
+ "unstable_source/inductor_cpp_wrapper_tutorial.rst",
+ "unstable_source/inductor_windows.rst",
+ "unstable_source/maskedtensor_advanced_semantics.py",
+ "unstable_source/max_autotune_on_CPU_tutorial.rst",
+ "unstable_source/vmap_recipe.py",
+ "recipes_source/README.txt",
+ "recipes_source/compiling_optimizer.rst",
+ "recipes_source/compiling_optimizer_lr_scheduler.py",
+ "recipes_source/distributed_optim_torchscript.rst",
+ "recipes_source/foreach_map.py",
+ "recipes_source/profile_with_itt.rst",
+ "recipes_source/recipes/Captum_Recipe.py",
+ "recipes_source/recipes/benchmark.py",
+ "recipes_source/recipes/changing_default_device.py",
+ "recipes_source/recipes/defining_a_neural_network.py",
+ "recipes_source/recipes/tensorboard_with_pytorch.py",
+ "recipes_source/recipes/timer_quick_start.py",
+ "recipes_source/recipes/tuning_guide.py",
+ "recipes_source/recipes/warmstarting_model_using_parameters_from_a_different_model.py",
+ "recipes_source/recipes/what_is_state_dict.py",
+ "recipes_source/torch_compile_caching_tutorial.rst",
+ "recipes_source/torch_compile_torch_function_modes.py",
+ "recipes_source/torch_compile_user_defined_triton_kernel_tutorial.py",
+ "recipes_source/torch_compiler_set_stance_tutorial.py",
+ "recipes_source/torch_export_aoti_python.py",
+ "recipes_source/xeon_run_cpu.rst",
+ "advanced_source/cpp_export.rst",
+ "advanced_source/torch-script-parallelism.rst",
+ "advanced_source/torch_script_custom_classes.rst",
+ "advanced_source/torch_script_custom_ops.rst",
+ "recipes_source/torchscript_inference.rst",
+]
+init_command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--lint-name=grep_linter.py',
+ '--lint-link=https://raw.githubusercontent.com/pytorch/pytorch/4805a6ead6f1e7f32351056e2602be4e908f69b7/tools/linter/adapters/grep_linter.py',
+ '--',
+ '--dry-run={{DRYRUN}}',
+]
+command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--run-lint',
+ '--lint-name=grep_linter.py',
+ '--',
+ '--pattern=[[:blank:]]$',
+ '--linter-name=SPACES',
+ '--error-name=trailing spaces',
+ '--replace-pattern=s/[[:blank:]]+$//',
+ """--error-description=\
+ This line has trailing spaces; please remove them.\
+ """,
+ '--',
+ '@{{PATHSFILE}}'
+]
+
+[[linter]]
+code = 'TABS'
+include_patterns = ['**']
+exclude_patterns = [
+ "_static/**/*", # Contains some files that should usually not be linted
+ ".lintrunner.toml", # Ironically needs to contain the tab character to find in other files
+ "Makefile", # Wants tabs for indentationo
+ # All files below this should be checked and either removed from the
+ # exclusion list by fixing them or have a reason to be excluded.
+ "advanced_source/README.txt",
+ "advanced_source/cpp_frontend.rst",
+ "advanced_source/torch_script_custom_ops.rst",
+ "beginner_source/README.txt",
+ "beginner_source/basics/tensorqs_tutorial.py",
+ "beginner_source/blitz/README.txt",
+ "beginner_source/blitz/tensor_tutorial.py",
+ "beginner_source/hybrid_frontend/README.txt",
+ "beginner_source/nlp/README.txt",
+ "beginner_source/nlp/pytorch_tutorial.py",
+ "intermediate_source/README.txt",
+ "intermediate_source/TP_tutorial.rst",
+ "intermediate_source/inductor_debug_cpu.py",
+ "unstable_source/README.txt",
+ "recipes_source/README.txt",
+ "recipes_source/recipes/README.txt",
+ "recipes_source/xeon_run_cpu.rst",
+]
+init_command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--lint-name=grep_linter.py',
+ '--lint-link=https://raw.githubusercontent.com/pytorch/pytorch/4805a6ead6f1e7f32351056e2602be4e908f69b7/tools/linter/adapters/grep_linter.py',
+ '--',
+ '--dry-run={{DRYRUN}}',
+]
+command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--run-lint',
+ '--lint-name=grep_linter.py',
+ '--',
+ # @lint-ignore TXT2
+ '--pattern= ',
+ '--linter-name=TABS',
+ '--error-name=saw some tabs',
+ '--replace-pattern=s/\t/ /',
+ """--error-description=\
+ This line has tabs; please replace them with spaces.\
+ """,
+ '--',
+ '@{{PATHSFILE}}'
+]
+
+[[linter]]
+code = 'NEWLINE'
+include_patterns=['**']
+exclude_patterns=[
+ "_static/**/*", # Contains some files that should usually not be linted
+ # All files below this should be checked and either removed from the
+ # exclusion list by fixing them or have a reason to be excluded.
+ "advanced_source/extend_dispatcher.rst",
+ "advanced_source/neural_style_tutorial.py",
+ "advanced_source/sharding.rst",
+ "advanced_source/torch_script_custom_classes/custom_class_project/custom_test.py",
+ "advanced_source/transformer__timeseries_cpp_tutorial/transformer_timeseries.cpp",
+ "beginner_source/blitz/README.txt",
+ "beginner_source/dcgan_faces_tutorial.py",
+ "beginner_source/hta_trace_diff_tutorial.rst",
+ "beginner_source/hybrid_frontend/README.txt",
+ "beginner_source/nlp/pytorch_tutorial.py",
+ "beginner_source/template_tutorial.py",
+ "beginner_source/transfer_learning_tutorial.py",
+ "intermediate_source/custom_function_conv_bn_tutorial.py",
+ "intermediate_source/custom_function_double_backward_tutorial.rst",
+ "intermediate_source/forced_alignment_with_torchaudio_tutorial.rst",
+ "intermediate_source/nlp_from_scratch_index.rst",
+ "intermediate_source/pipeline_tutorial.rst",
+ "recipes_source/README.txt",
+ "recipes_source/script_optimized.rst",
+ "recipes_source/torch_compile_caching_configuration_tutorial.rst",
+ "recipes_source/torch_compile_caching_tutorial.rst",
+]
+init_command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--lint-name=newlines_linter.py',
+ '--lint-link=https://raw.githubusercontent.com/pytorch/pytorch/4805a6ead6f1e7f32351056e2602be4e908f69b7/tools/linter/adapters/newlines_linter.py',
+ '--',
+ '--dry-run={{DRYRUN}}',
+]
+command = [
+ 'python3',
+ 'tools/linter/adapters/run_from_link.py',
+ '--run-lint',
+ '--lint-name=newlines_linter.py',
+ '--',
+ '@{{PATHSFILE}}',
+]
+is_formatter = true
diff --git a/.lycheeignore b/.lycheeignore
new file mode 100644
index 00000000000..fc1e3f1fa85
--- /dev/null
+++ b/.lycheeignore
@@ -0,0 +1,17 @@
+# Used for links to be ignored during the link check.
+# Add link to file along with comment as to why it should be ignored
+
+#Example link in some of the tutorials that should be ignored
+file:///f:/libtmp/some_file
+
+#Ignore links with "file:///" to catch any other example links
+file:\/\/\/.*
+
+# Ignore colab link in the setting of conf.py
+https://pytorch.org/tutorials/beginner/colab/n
+
+# Ignore local host link from intermediate_source/tensorboard_tutorial.rst
+http://localhost:6006
+
+# Ignore local host link from advanced_source/cpp_frontend.rst
+https://www.uber.com/blog/deep-neuroevolution/
diff --git a/.pyspelling.yml b/.pyspelling.yml
index 1afe6dbb45e..bce797e6559 100644
--- a/.pyspelling.yml
+++ b/.pyspelling.yml
@@ -2,10 +2,7 @@ spellchecker: aspell
matrix:
- name: python
sources:
- - beginner_source/*.py
- - intermediate_source/*.py
- - advanced_source/*.py
- - recipes_source/*/*.py
+ - "**/*.py"
dictionary:
wordlists:
- en-wordlist.txt
@@ -56,7 +53,7 @@ matrix:
- pyspelling.filters.url:
- name: reST
sources:
- - beginner_source/*.rst
+ - "**/*.rst"
dictionary:
wordlists:
- en-wordlist.txt
@@ -119,3 +116,48 @@ matrix:
- open: '\.\.\s+(image|include|only)::'
close: '$'
- pyspelling.filters.url:
+- name: markdown
+ sources:
+ - '**/*.md'
+ dictionary:
+ wordlists:
+ - en-wordlist.txt
+ pipeline:
+ - pyspelling.filters.markdown:
+ markdown_extensions:
+ - markdown.extensions.extra:
+ - markdown.extensions.admonition:
+ - markdown.extensions.codehilite:
+ - markdown.extensions.meta:
+ - markdown.extensions.tables:
+ - markdown.extensions.toc:
+ - pyspelling.filters.html:
+ comments: false
+ ignores:
+ - code
+ - pre
+ - tt
+ - img
+ - a
+ - table
+ - thead
+ - tbody
+ - th
+ - tr
+ - td
+ - pyspelling.filters.context:
+ context_visible_first: true
+ delimiters:
+ # Ignore code blocks
+ - open: '```[a-z]*\n'
+ close: '```\n'
+ # Ignore inline code
+ - open: '`'
+ close: '`'
+ # Ignore links
+ - open: '\[([^]]*)\]'
+ close: '\([^)]*\)'
+ # Ignore HTML comments
+ - open: ''
+ - pyspelling.filters.url:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 5a3ab42272c..0879eeebdff 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -71,8 +71,7 @@ There are three types of tutorial content that we host on
reStructuredText files. The build system only converts them into HTML;
the code in them does not run on build. These tutorials are easier to
create and maintain but they do not provide an interactive experience.
- An example is the [Dynamic Quantization
- tutorial](https://pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html).
+
* **Recipes** are tutorials that provide bite-sized, actionable
examples of how to use specific features, which differentiates them
@@ -162,7 +161,7 @@ Write for a global audience with an instructive and directive voice.
- PyTorch has a global audience; use clear, easy to understand
language. Avoid idioms or other figures of speech.
-- To keep your instructions concise, use
+- To keep your instructions concise, use
[active voice](https://writing.wisc.edu/handbook/style/ccs_activevoice/) as much as possible.
- For a short guide on the essentials of writing style,
[The Elements of Style](https://www.gutenberg.org/files/37134/37134-h/37134-h.htm)
@@ -218,9 +217,8 @@ described in the preceding sections:
- [NLP From Scratch: Generating Names with a Character-Level RNN
Tutorial](https://pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html)
-If you are creating a recipe, we recommend that you use [this
-template](https://github.com/pytorch/tutorials/blob/tutorials_refresh/recipes_source/recipes/example_recipe.py)
-as a guide.
+If you are creating a recipe, [this is a good
+example.](https://github.com/pytorch/tutorials/blob/main/recipes_source/recipes/what_is_state_dict.py)
# Submission Process #
@@ -261,12 +259,12 @@ For Python files, our CI system runs your code during each build.
In order for your tutorial to appear on the website, and through tag
search, you need to include it in `index.rst`, or for recipes, in
-`recipes_index.rst`.
+`recipes_index.rst`.
1. Open the relevant file
[`index.rst`](https://github.com/pytorch/tutorials/blob/main/index.rst)
or
- [`recipes_index.rst`](https://github.com/pytorch/tutorials/blob/main/recipes_source/recipes_index.rst)
+ [`recipes_index.rst`](https://github.com/pytorch/tutorials/blob/main/recipes_index.rst)
1. Add a _card_ in reStructuredText format similar to the following:
```
@@ -358,13 +356,12 @@ Submit the changes as a PR to the main branch of
1. Address all feedback comments from your reviewers.
1. Make sure all CI checks are passing.
-Once you submit your PR, you can see a generated Netlify preview of your
-build. You can see an example Netlify preview at the following URL:
-
-> ' +
- '' +
- Documentation.gettext("Hide Search Matches") +
- "
"
- )
- );
- },
-
- /**
- * helper function to hide the search marks again
- */
- hideSearchWords: () => {
- document
- .querySelectorAll("#searchbox .highlight-link")
- .forEach((el) => el.remove());
- document
- .querySelectorAll("span.highlighted")
- .forEach((el) => el.classList.remove("highlighted"));
- const url = new URL(window.location);
- url.searchParams.delete("highlight");
- window.history.replaceState({}, "", url);
- },
-
- /**
- * helper function to focus on search bar
- */
- focusSearchBar: () => {
- document.querySelectorAll("input[name=q]")[0]?.focus();
- },
-
- /**
- * Initialise the domain index toggle buttons
- */
- initDomainIndexTable: () => {
- const toggler = (el) => {
- const idNumber = el.id.substr(7);
- const toggledRows = document.querySelectorAll(`tr.cg-${idNumber}`);
- if (el.src.substr(-9) === "minus.png") {
- el.src = `${el.src.substr(0, el.src.length - 9)}plus.png`;
- toggledRows.forEach((el) => (el.style.display = "none"));
- } else {
- el.src = `${el.src.substr(0, el.src.length - 8)}minus.png`;
- toggledRows.forEach((el) => (el.style.display = ""));
- }
- };
-
- const togglerElements = document.querySelectorAll("img.toggler");
- togglerElements.forEach((el) =>
- el.addEventListener("click", (event) => toggler(event.currentTarget))
- );
- togglerElements.forEach((el) => (el.style.display = ""));
- if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) togglerElements.forEach(toggler);
- },
-
- initOnKeyListeners: () => {
- // only install a listener if it is really needed
- if (
- !DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS &&
- !DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS
- )
- return;
-
- const blacklistedElements = new Set([
- "TEXTAREA",
- "INPUT",
- "SELECT",
- "BUTTON",
- ]);
- document.addEventListener("keydown", (event) => {
- if (blacklistedElements.has(document.activeElement.tagName)) return; // bail for input elements
- if (event.altKey || event.ctrlKey || event.metaKey) return; // bail with special keys
-
- if (!event.shiftKey) {
- switch (event.key) {
- case "ArrowLeft":
- if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break;
-
- const prevLink = document.querySelector('link[rel="prev"]');
- if (prevLink && prevLink.href) {
- window.location.href = prevLink.href;
- event.preventDefault();
- }
- break;
- case "ArrowRight":
- if (!DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) break;
-
- const nextLink = document.querySelector('link[rel="next"]');
- if (nextLink && nextLink.href) {
- window.location.href = nextLink.href;
- event.preventDefault();
- }
- break;
- case "Escape":
- if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break;
- Documentation.hideSearchWords();
- event.preventDefault();
- }
- }
-
- // some keyboard layouts may need Shift to get /
- switch (event.key) {
- case "/":
- if (!DOCUMENTATION_OPTIONS.ENABLE_SEARCH_SHORTCUTS) break;
- Documentation.focusSearchBar();
- event.preventDefault();
- }
- });
- },
-};
-
-// quick alias for translations
-const _ = Documentation.gettext;
-
-_ready(Documentation.init);
diff --git a/_static/documentation_options.js b/_static/documentation_options.js
deleted file mode 100755
index a9214d61b9c..00000000000
--- a/_static/documentation_options.js
+++ /dev/null
@@ -1,9 +0,0 @@
-var DOCUMENTATION_OPTIONS = {
- URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
- VERSION: '0.5.0a0+a24163a',
- LANGUAGE: 'None',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
-};
\ No newline at end of file
diff --git a/_static/img/bert.png b/_static/img/bert.png
deleted file mode 100644
index 6e23a8acfd3..00000000000
Binary files a/_static/img/bert.png and /dev/null differ
diff --git a/_static/img/cat_224x224.jpg b/_static/img/cat_224x224.jpg
deleted file mode 100755
index 05660ce53f9..00000000000
Binary files a/_static/img/cat_224x224.jpg and /dev/null differ
diff --git a/_static/img/cat_resized.jpg b/_static/img/cat_resized.jpg
deleted file mode 100644
index c7746e65308..00000000000
Binary files a/_static/img/cat_resized.jpg and /dev/null differ
diff --git a/_static/img/cat_superres_with_ort.jpg b/_static/img/cat_superres_with_ort.jpg
deleted file mode 100644
index 7e4143c3e79..00000000000
Binary files a/_static/img/cat_superres_with_ort.jpg and /dev/null differ
diff --git a/_static/img/compare_output.png b/_static/img/compare_output.png
deleted file mode 100644
index 4ece4d11483..00000000000
Binary files a/_static/img/compare_output.png and /dev/null differ
diff --git a/_static/img/compare_stub.png b/_static/img/compare_stub.png
deleted file mode 100644
index 8140a99b182..00000000000
Binary files a/_static/img/compare_stub.png and /dev/null differ
diff --git a/_static/img/compiled_autograd/call_hook_node.png b/_static/img/compiled_autograd/call_hook_node.png
new file mode 100644
index 00000000000..3e094cf6f73
Binary files /dev/null and b/_static/img/compiled_autograd/call_hook_node.png differ
diff --git a/_static/img/compiled_autograd/entire_verbose_log.png b/_static/img/compiled_autograd/entire_verbose_log.png
new file mode 100644
index 00000000000..4ce2b8538ee
Binary files /dev/null and b/_static/img/compiled_autograd/entire_verbose_log.png differ
diff --git a/_static/img/compiled_autograd/recompile_due_to_dynamic.png b/_static/img/compiled_autograd/recompile_due_to_dynamic.png
new file mode 100644
index 00000000000..41ae56acf2d
Binary files /dev/null and b/_static/img/compiled_autograd/recompile_due_to_dynamic.png differ
diff --git a/_static/img/compiled_autograd/recompile_due_to_node.png b/_static/img/compiled_autograd/recompile_due_to_node.png
new file mode 100644
index 00000000000..800a1784587
Binary files /dev/null and b/_static/img/compiled_autograd/recompile_due_to_node.png differ
diff --git a/_static/img/distributed/fsdp_implicit.png b/_static/img/distributed/fsdp_implicit.png
new file mode 100644
index 00000000000..85b19b7e72e
Binary files /dev/null and b/_static/img/distributed/fsdp_implicit.png differ
diff --git a/_static/img/distributed/tcpstore_barrier_time.png b/_static/img/distributed/tcpstore_barrier_time.png
new file mode 100644
index 00000000000..5ece3a7471d
Binary files /dev/null and b/_static/img/distributed/tcpstore_barrier_time.png differ
diff --git a/_static/img/distributed/tcpstore_init_time.png b/_static/img/distributed/tcpstore_init_time.png
new file mode 100644
index 00000000000..df514b4dc48
Binary files /dev/null and b/_static/img/distributed/tcpstore_init_time.png differ
diff --git a/_static/img/install_msvc.png b/_static/img/install_msvc.png
new file mode 100644
index 00000000000..fce73207a80
Binary files /dev/null and b/_static/img/install_msvc.png differ
diff --git a/_static/img/itt_tutorial/vtune_xpu_config.png b/_static/img/itt_tutorial/vtune_xpu_config.png
new file mode 100644
index 00000000000..80dd1812d26
Binary files /dev/null and b/_static/img/itt_tutorial/vtune_xpu_config.png differ
diff --git a/_static/img/itt_tutorial/vtune_xpu_timeline.png b/_static/img/itt_tutorial/vtune_xpu_timeline.png
new file mode 100644
index 00000000000..43818cf105c
Binary files /dev/null and b/_static/img/itt_tutorial/vtune_xpu_timeline.png differ
diff --git a/_static/img/onnx/custom_addandround_function.png b/_static/img/onnx/custom_addandround_function.png
deleted file mode 100644
index a0c7000161e..00000000000
Binary files a/_static/img/onnx/custom_addandround_function.png and /dev/null differ
diff --git a/_static/img/onnx/custom_addandround_model.png b/_static/img/onnx/custom_addandround_model.png
deleted file mode 100644
index 793d8cfbb5d..00000000000
Binary files a/_static/img/onnx/custom_addandround_model.png and /dev/null differ
diff --git a/_static/img/onnx/custom_aten_add_function.png b/_static/img/onnx/custom_aten_add_function.png
deleted file mode 100644
index d9f927ce707..00000000000
Binary files a/_static/img/onnx/custom_aten_add_function.png and /dev/null differ
diff --git a/_static/img/onnx/custom_aten_add_model.png b/_static/img/onnx/custom_aten_add_model.png
deleted file mode 100644
index e5ef1c71742..00000000000
Binary files a/_static/img/onnx/custom_aten_add_model.png and /dev/null differ
diff --git a/_static/img/onnx/custom_aten_gelu_function.png b/_static/img/onnx/custom_aten_gelu_function.png
deleted file mode 100644
index 5cb573e7dcb..00000000000
Binary files a/_static/img/onnx/custom_aten_gelu_function.png and /dev/null differ
diff --git a/_static/img/onnx/custom_aten_gelu_model.png b/_static/img/onnx/custom_aten_gelu_model.png
deleted file mode 100644
index 6bc46337b48..00000000000
Binary files a/_static/img/onnx/custom_aten_gelu_model.png and /dev/null differ
diff --git a/_static/img/onnx/image_classifier_onnx_model_on_netron_web_ui.png b/_static/img/onnx/image_classifier_onnx_model_on_netron_web_ui.png
new file mode 100644
index 00000000000..6430e4943ff
Binary files /dev/null and b/_static/img/onnx/image_classifier_onnx_model_on_netron_web_ui.png differ
diff --git a/_static/img/onnx/image_clossifier_onnx_modelon_netron_web_ui.png b/_static/img/onnx/image_clossifier_onnx_modelon_netron_web_ui.png
deleted file mode 100755
index 0c29c168798..00000000000
Binary files a/_static/img/onnx/image_clossifier_onnx_modelon_netron_web_ui.png and /dev/null differ
diff --git a/_static/img/pinmem/pinmem.png b/_static/img/pinmem/pinmem.png
new file mode 100644
index 00000000000..9d84e9d229d
Binary files /dev/null and b/_static/img/pinmem/pinmem.png differ
diff --git a/_static/img/pinmem/trace_streamed0_pinned0.png b/_static/img/pinmem/trace_streamed0_pinned0.png
new file mode 100644
index 00000000000..dedac997b0b
Binary files /dev/null and b/_static/img/pinmem/trace_streamed0_pinned0.png differ
diff --git a/_static/img/pinmem/trace_streamed0_pinned1.png b/_static/img/pinmem/trace_streamed0_pinned1.png
new file mode 100644
index 00000000000..2d5ff462e1a
Binary files /dev/null and b/_static/img/pinmem/trace_streamed0_pinned1.png differ
diff --git a/_static/img/pinmem/trace_streamed1_pinned0.png b/_static/img/pinmem/trace_streamed1_pinned0.png
new file mode 100644
index 00000000000..130182a1978
Binary files /dev/null and b/_static/img/pinmem/trace_streamed1_pinned0.png differ
diff --git a/_static/img/pinmem/trace_streamed1_pinned1.png b/_static/img/pinmem/trace_streamed1_pinned1.png
new file mode 100644
index 00000000000..c596fcdb691
Binary files /dev/null and b/_static/img/pinmem/trace_streamed1_pinned1.png differ
diff --git a/_static/img/python_extension_autoload_impl.png b/_static/img/python_extension_autoload_impl.png
new file mode 100644
index 00000000000..64e18fc7b4b
Binary files /dev/null and b/_static/img/python_extension_autoload_impl.png differ
diff --git a/_static/img/quant_asym.png b/_static/img/quant_asym.png
deleted file mode 100644
index 9dc43817a59..00000000000
Binary files a/_static/img/quant_asym.png and /dev/null differ
diff --git a/_static/img/quantized_transfer_learning.png b/_static/img/quantized_transfer_learning.png
deleted file mode 100644
index c138cbdb0c1..00000000000
Binary files a/_static/img/quantized_transfer_learning.png and /dev/null differ
diff --git a/_static/img/shadow.png b/_static/img/shadow.png
deleted file mode 100644
index e09d0b87f01..00000000000
Binary files a/_static/img/shadow.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-BERT.png b/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-BERT.png
deleted file mode 100644
index 34bbf8c7bdf..00000000000
Binary files a/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-BERT.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-an-LSTM-Word-Language-Model.png b/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-an-LSTM-Word-Language-Model.png
deleted file mode 100644
index 986efaa3f88..00000000000
Binary files a/_static/img/thumbnails/cropped/experimental-Dynamic-Quantization-on-an-LSTM-Word-Language-Model.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/graph-mode-dynamic-bert.png b/_static/img/thumbnails/cropped/graph-mode-dynamic-bert.png
deleted file mode 100644
index 34bbf8c7bdf..00000000000
Binary files a/_static/img/thumbnails/cropped/graph-mode-dynamic-bert.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/mobile.png b/_static/img/thumbnails/cropped/mobile.png
deleted file mode 100644
index 12dc917519c..00000000000
Binary files a/_static/img/thumbnails/cropped/mobile.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/optional-Exporting-a-Model-from-PyTorch-to-ONNX-and-Running-it-using-ONNX-Runtime.png b/_static/img/thumbnails/cropped/optional-Exporting-a-Model-from-PyTorch-to-ONNX-and-Running-it-using-ONNX-Runtime.png
deleted file mode 100644
index 00156df042e..00000000000
Binary files a/_static/img/thumbnails/cropped/optional-Exporting-a-Model-from-PyTorch-to-ONNX-and-Running-it-using-ONNX-Runtime.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/understanding_leaf_vs_nonleaf.png b/_static/img/thumbnails/cropped/understanding_leaf_vs_nonleaf.png
new file mode 100644
index 00000000000..0590cf227d9
Binary files /dev/null and b/_static/img/thumbnails/cropped/understanding_leaf_vs_nonleaf.png differ
diff --git a/_static/img/thumbnails/cropped/using-dynamic-post-training-quantization.png b/_static/img/thumbnails/cropped/using-dynamic-post-training-quantization.png
deleted file mode 100644
index 6ce22e4862a..00000000000
Binary files a/_static/img/thumbnails/cropped/using-dynamic-post-training-quantization.png and /dev/null differ
diff --git a/_static/img/thumbnails/cropped/visualizing_gradients_tutorial.png b/_static/img/thumbnails/cropped/visualizing_gradients_tutorial.png
new file mode 100644
index 00000000000..6ff6d97f2e2
Binary files /dev/null and b/_static/img/thumbnails/cropped/visualizing_gradients_tutorial.png differ
diff --git a/_static/img/tiatoolbox_tutorial/read_bounds_tissue.webp b/_static/img/tiatoolbox_tutorial/read_bounds_tissue.webp
deleted file mode 100644
index 5a1ca81e07d..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/read_bounds_tissue.webp and /dev/null differ
diff --git a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_001.png b/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_001.png
deleted file mode 100644
index fafd95768a1..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_001.png and /dev/null differ
diff --git a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_002.png b/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_002.png
deleted file mode 100644
index fd6f7aba1f4..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_002.png and /dev/null differ
diff --git a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_003.png b/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_003.png
deleted file mode 100644
index 8feda69de2d..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_003.png and /dev/null differ
diff --git a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_004.png b/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_004.png
deleted file mode 100644
index 8feda69de2d..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_004.png and /dev/null differ
diff --git a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_005.png b/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_005.png
deleted file mode 100644
index e17e03812ce..00000000000
Binary files a/_static/img/tiatoolbox_tutorial/tiatoolbox_tutorial_005.png and /dev/null differ
diff --git a/_static/img/trace_xpu_img.png b/_static/img/trace_xpu_img.png
new file mode 100644
index 00000000000..2eca0a78cb6
Binary files /dev/null and b/_static/img/trace_xpu_img.png differ
diff --git a/_static/img/understanding_leaf_vs_nonleaf/comp-graph-1.png b/_static/img/understanding_leaf_vs_nonleaf/comp-graph-1.png
new file mode 100644
index 00000000000..1fa3d80d339
Binary files /dev/null and b/_static/img/understanding_leaf_vs_nonleaf/comp-graph-1.png differ
diff --git a/_static/img/understanding_leaf_vs_nonleaf/comp-graph-2.png b/_static/img/understanding_leaf_vs_nonleaf/comp-graph-2.png
new file mode 100644
index 00000000000..3f76deab3bf
Binary files /dev/null and b/_static/img/understanding_leaf_vs_nonleaf/comp-graph-2.png differ
diff --git a/_static/jquery-3.2.1.js b/_static/jquery-3.2.1.js
deleted file mode 100755
index 2cbd2ab50e7..00000000000
--- a/_static/jquery-3.2.1.js
+++ /dev/null
@@ -1,10253 +0,0 @@
-/*!
- * jQuery JavaScript Library v3.2.1
- * https://jquery.com/
- *
- * Includes Sizzle.js
- * https://sizzlejs.com/
- *
- * Copyright JS Foundation and other contributors
- * Released under the MIT license
- * https://jquery.org/license
- *
- * Date: 2017-03-20T18:59Z
- */
-( function( global, factory ) {
-
- "use strict";
-
- if ( typeof module === "object" && typeof module.exports === "object" ) {
-
- // For CommonJS and CommonJS-like environments where a proper `window`
- // is present, execute the factory and get jQuery.
- // For environments that do not have a `window` with a `document`
- // (such as Node.js), expose a factory as module.exports.
- // This accentuates the need for the creation of a real `window`.
- // e.g. var jQuery = require("jquery")(window);
- // See ticket #14549 for more info.
- module.exports = global.document ?
- factory( global, true ) :
- function( w ) {
- if ( !w.document ) {
- throw new Error( "jQuery requires a window with a document" );
- }
- return factory( w );
- };
- } else {
- factory( global );
- }
-
-// Pass this if window is not defined yet
-} )( typeof window !== "undefined" ? window : this, function( window, noGlobal ) {
-
-// Edge <= 12 - 13+, Firefox <=18 - 45+, IE 10 - 11, Safari 5.1 - 9+, iOS 6 - 9.1
-// throw exceptions when non-strict code (e.g., ASP.NET 4.5) accesses strict mode
-// arguments.callee.caller (trac-13335). But as of jQuery 3.0 (2016), strict mode should be common
-// enough that all such attempts are guarded in a try block.
-"use strict";
-
-var arr = [];
-
-var document = window.document;
-
-var getProto = Object.getPrototypeOf;
-
-var slice = arr.slice;
-
-var concat = arr.concat;
-
-var push = arr.push;
-
-var indexOf = arr.indexOf;
-
-var class2type = {};
-
-var toString = class2type.toString;
-
-var hasOwn = class2type.hasOwnProperty;
-
-var fnToString = hasOwn.toString;
-
-var ObjectFunctionString = fnToString.call( Object );
-
-var support = {};
-
-
-
- function DOMEval( code, doc ) {
- doc = doc || document;
-
- var script = doc.createElement( "script" );
-
- script.text = code;
- doc.head.appendChild( script ).parentNode.removeChild( script );
- }
-/* global Symbol */
-// Defining this global in .eslintrc.json would create a danger of using the global
-// unguarded in another place, it seems safer to define global only for this module
-
-
-
-var
- version = "3.2.1",
-
- // Define a local copy of jQuery
- jQuery = function( selector, context ) {
-
- // The jQuery object is actually just the init constructor 'enhanced'
- // Need init if jQuery is called (just allow error to be thrown if not included)
- return new jQuery.fn.init( selector, context );
- },
-
- // Support: Android <=4.0 only
- // Make sure we trim BOM and NBSP
- rtrim = /^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,
-
- // Matches dashed string for camelizing
- rmsPrefix = /^-ms-/,
- rdashAlpha = /-([a-z])/g,
-
- // Used by jQuery.camelCase as callback to replace()
- fcamelCase = function( all, letter ) {
- return letter.toUpperCase();
- };
-
-jQuery.fn = jQuery.prototype = {
-
- // The current version of jQuery being used
- jquery: version,
-
- constructor: jQuery,
-
- // The default length of a jQuery object is 0
- length: 0,
-
- toArray: function() {
- return slice.call( this );
- },
-
- // Get the Nth element in the matched element set OR
- // Get the whole matched element set as a clean array
- get: function( num ) {
-
- // Return all the elements in a clean array
- if ( num == null ) {
- return slice.call( this );
- }
-
- // Return just the one element from the set
- return num < 0 ? this[ num + this.length ] : this[ num ];
- },
-
- // Take an array of elements and push it onto the stack
- // (returning the new matched element set)
- pushStack: function( elems ) {
-
- // Build a new jQuery matched element set
- var ret = jQuery.merge( this.constructor(), elems );
-
- // Add the old object onto the stack (as a reference)
- ret.prevObject = this;
-
- // Return the newly-formed element set
- return ret;
- },
-
- // Execute a callback for every element in the matched set.
- each: function( callback ) {
- return jQuery.each( this, callback );
- },
-
- map: function( callback ) {
- return this.pushStack( jQuery.map( this, function( elem, i ) {
- return callback.call( elem, i, elem );
- } ) );
- },
-
- slice: function() {
- return this.pushStack( slice.apply( this, arguments ) );
- },
-
- first: function() {
- return this.eq( 0 );
- },
-
- last: function() {
- return this.eq( -1 );
- },
-
- eq: function( i ) {
- var len = this.length,
- j = +i + ( i < 0 ? len : 0 );
- return this.pushStack( j >= 0 && j < len ? [ this[ j ] ] : [] );
- },
-
- end: function() {
- return this.prevObject || this.constructor();
- },
-
- // For internal use only.
- // Behaves like an Array's method, not like a jQuery method.
- push: push,
- sort: arr.sort,
- splice: arr.splice
-};
-
-jQuery.extend = jQuery.fn.extend = function() {
- var options, name, src, copy, copyIsArray, clone,
- target = arguments[ 0 ] || {},
- i = 1,
- length = arguments.length,
- deep = false;
-
- // Handle a deep copy situation
- if ( typeof target === "boolean" ) {
- deep = target;
-
- // Skip the boolean and the target
- target = arguments[ i ] || {};
- i++;
- }
-
- // Handle case when target is a string or something (possible in deep copy)
- if ( typeof target !== "object" && !jQuery.isFunction( target ) ) {
- target = {};
- }
-
- // Extend jQuery itself if only one argument is passed
- if ( i === length ) {
- target = this;
- i--;
- }
-
- for ( ; i < length; i++ ) {
-
- // Only deal with non-null/undefined values
- if ( ( options = arguments[ i ] ) != null ) {
-
- // Extend the base object
- for ( name in options ) {
- src = target[ name ];
- copy = options[ name ];
-
- // Prevent never-ending loop
- if ( target === copy ) {
- continue;
- }
-
- // Recurse if we're merging plain objects or arrays
- if ( deep && copy && ( jQuery.isPlainObject( copy ) ||
- ( copyIsArray = Array.isArray( copy ) ) ) ) {
-
- if ( copyIsArray ) {
- copyIsArray = false;
- clone = src && Array.isArray( src ) ? src : [];
-
- } else {
- clone = src && jQuery.isPlainObject( src ) ? src : {};
- }
-
- // Never move original objects, clone them
- target[ name ] = jQuery.extend( deep, clone, copy );
-
- // Don't bring in undefined values
- } else if ( copy !== undefined ) {
- target[ name ] = copy;
- }
- }
- }
- }
-
- // Return the modified object
- return target;
-};
-
-jQuery.extend( {
-
- // Unique for each copy of jQuery on the page
- expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ),
-
- // Assume jQuery is ready without the ready module
- isReady: true,
-
- error: function( msg ) {
- throw new Error( msg );
- },
-
- noop: function() {},
-
- isFunction: function( obj ) {
- return jQuery.type( obj ) === "function";
- },
-
- isWindow: function( obj ) {
- return obj != null && obj === obj.window;
- },
-
- isNumeric: function( obj ) {
-
- // As of jQuery 3.0, isNumeric is limited to
- // strings and numbers (primitives or objects)
- // that can be coerced to finite numbers (gh-2662)
- var type = jQuery.type( obj );
- return ( type === "number" || type === "string" ) &&
-
- // parseFloat NaNs numeric-cast false positives ("")
- // ...but misinterprets leading-number strings, particularly hex literals ("0x...")
- // subtraction forces infinities to NaN
- !isNaN( obj - parseFloat( obj ) );
- },
-
- isPlainObject: function( obj ) {
- var proto, Ctor;
-
- // Detect obvious negatives
- // Use toString instead of jQuery.type to catch host objects
- if ( !obj || toString.call( obj ) !== "[object Object]" ) {
- return false;
- }
-
- proto = getProto( obj );
-
- // Objects with no prototype (e.g., `Object.create( null )`) are plain
- if ( !proto ) {
- return true;
- }
-
- // Objects with prototype are plain iff they were constructed by a global Object function
- Ctor = hasOwn.call( proto, "constructor" ) && proto.constructor;
- return typeof Ctor === "function" && fnToString.call( Ctor ) === ObjectFunctionString;
- },
-
- isEmptyObject: function( obj ) {
-
- /* eslint-disable no-unused-vars */
- // See https://github.com/eslint/eslint/issues/6125
- var name;
-
- for ( name in obj ) {
- return false;
- }
- return true;
- },
-
- type: function( obj ) {
- if ( obj == null ) {
- return obj + "";
- }
-
- // Support: Android <=2.3 only (functionish RegExp)
- return typeof obj === "object" || typeof obj === "function" ?
- class2type[ toString.call( obj ) ] || "object" :
- typeof obj;
- },
-
- // Evaluates a script in a global context
- globalEval: function( code ) {
- DOMEval( code );
- },
-
- // Convert dashed to camelCase; used by the css and data modules
- // Support: IE <=9 - 11, Edge 12 - 13
- // Microsoft forgot to hump their vendor prefix (#9572)
- camelCase: function( string ) {
- return string.replace( rmsPrefix, "ms-" ).replace( rdashAlpha, fcamelCase );
- },
-
- each: function( obj, callback ) {
- var length, i = 0;
-
- if ( isArrayLike( obj ) ) {
- length = obj.length;
- for ( ; i < length; i++ ) {
- if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) {
- break;
- }
- }
- } else {
- for ( i in obj ) {
- if ( callback.call( obj[ i ], i, obj[ i ] ) === false ) {
- break;
- }
- }
- }
-
- return obj;
- },
-
- // Support: Android <=4.0 only
- trim: function( text ) {
- return text == null ?
- "" :
- ( text + "" ).replace( rtrim, "" );
- },
-
- // results is for internal usage only
- makeArray: function( arr, results ) {
- var ret = results || [];
-
- if ( arr != null ) {
- if ( isArrayLike( Object( arr ) ) ) {
- jQuery.merge( ret,
- typeof arr === "string" ?
- [ arr ] : arr
- );
- } else {
- push.call( ret, arr );
- }
- }
-
- return ret;
- },
-
- inArray: function( elem, arr, i ) {
- return arr == null ? -1 : indexOf.call( arr, elem, i );
- },
-
- // Support: Android <=4.0 only, PhantomJS 1 only
- // push.apply(_, arraylike) throws on ancient WebKit
- merge: function( first, second ) {
- var len = +second.length,
- j = 0,
- i = first.length;
-
- for ( ; j < len; j++ ) {
- first[ i++ ] = second[ j ];
- }
-
- first.length = i;
-
- return first;
- },
-
- grep: function( elems, callback, invert ) {
- var callbackInverse,
- matches = [],
- i = 0,
- length = elems.length,
- callbackExpect = !invert;
-
- // Go through the array, only saving the items
- // that pass the validator function
- for ( ; i < length; i++ ) {
- callbackInverse = !callback( elems[ i ], i );
- if ( callbackInverse !== callbackExpect ) {
- matches.push( elems[ i ] );
- }
- }
-
- return matches;
- },
-
- // arg is for internal usage only
- map: function( elems, callback, arg ) {
- var length, value,
- i = 0,
- ret = [];
-
- // Go through the array, translating each of the items to their new values
- if ( isArrayLike( elems ) ) {
- length = elems.length;
- for ( ; i < length; i++ ) {
- value = callback( elems[ i ], i, arg );
-
- if ( value != null ) {
- ret.push( value );
- }
- }
-
- // Go through every key on the object,
- } else {
- for ( i in elems ) {
- value = callback( elems[ i ], i, arg );
-
- if ( value != null ) {
- ret.push( value );
- }
- }
- }
-
- // Flatten any nested arrays
- return concat.apply( [], ret );
- },
-
- // A global GUID counter for objects
- guid: 1,
-
- // Bind a function to a context, optionally partially applying any
- // arguments.
- proxy: function( fn, context ) {
- var tmp, args, proxy;
-
- if ( typeof context === "string" ) {
- tmp = fn[ context ];
- context = fn;
- fn = tmp;
- }
-
- // Quick check to determine if target is callable, in the spec
- // this throws a TypeError, but we will just return undefined.
- if ( !jQuery.isFunction( fn ) ) {
- return undefined;
- }
-
- // Simulated bind
- args = slice.call( arguments, 2 );
- proxy = function() {
- return fn.apply( context || this, args.concat( slice.call( arguments ) ) );
- };
-
- // Set the guid of unique handler to the same of original handler, so it can be removed
- proxy.guid = fn.guid = fn.guid || jQuery.guid++;
-
- return proxy;
- },
-
- now: Date.now,
-
- // jQuery.support is not used in Core but other projects attach their
- // properties to it so it needs to exist.
- support: support
-} );
-
-if ( typeof Symbol === "function" ) {
- jQuery.fn[ Symbol.iterator ] = arr[ Symbol.iterator ];
-}
-
-// Populate the class2type map
-jQuery.each( "Boolean Number String Function Array Date RegExp Object Error Symbol".split( " " ),
-function( i, name ) {
- class2type[ "[object " + name + "]" ] = name.toLowerCase();
-} );
-
-function isArrayLike( obj ) {
-
- // Support: real iOS 8.2 only (not reproducible in simulator)
- // `in` check used to prevent JIT error (gh-2145)
- // hasOwn isn't used here due to false negatives
- // regarding Nodelist length in IE
- var length = !!obj && "length" in obj && obj.length,
- type = jQuery.type( obj );
-
- if ( type === "function" || jQuery.isWindow( obj ) ) {
- return false;
- }
-
- return type === "array" || length === 0 ||
- typeof length === "number" && length > 0 && ( length - 1 ) in obj;
-}
-var Sizzle =
-/*!
- * Sizzle CSS Selector Engine v2.3.3
- * https://sizzlejs.com/
- *
- * Copyright jQuery Foundation and other contributors
- * Released under the MIT license
- * https://jquery.org/license
- *
- * Date: 2016-08-08
- */
-(function( window ) {
-
-var i,
- support,
- Expr,
- getText,
- isXML,
- tokenize,
- compile,
- select,
- outermostContext,
- sortInput,
- hasDuplicate,
-
- // Local document vars
- setDocument,
- document,
- docElem,
- documentIsHTML,
- rbuggyQSA,
- rbuggyMatches,
- matches,
- contains,
-
- // Instance-specific data
- expando = "sizzle" + 1 * new Date(),
- preferredDoc = window.document,
- dirruns = 0,
- done = 0,
- classCache = createCache(),
- tokenCache = createCache(),
- compilerCache = createCache(),
- sortOrder = function( a, b ) {
- if ( a === b ) {
- hasDuplicate = true;
- }
- return 0;
- },
-
- // Instance methods
- hasOwn = ({}).hasOwnProperty,
- arr = [],
- pop = arr.pop,
- push_native = arr.push,
- push = arr.push,
- slice = arr.slice,
- // Use a stripped-down indexOf as it's faster than native
- // https://jsperf.com/thor-indexof-vs-for/5
- indexOf = function( list, elem ) {
- var i = 0,
- len = list.length;
- for ( ; i < len; i++ ) {
- if ( list[i] === elem ) {
- return i;
- }
- }
- return -1;
- },
-
- booleans = "checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|ismap|loop|multiple|open|readonly|required|scoped",
-
- // Regular expressions
-
- // https://www.w3.org/TR/css3-selectors/#whitespace
- whitespace = "[\\x20\\t\\r\\n\\f]",
-
- // https://www.w3.org/TR/CSS21/syndata.html#value-def-identifier
- identifier = "(?:\\\\.|[\\w-]|[^\0-\\xa0])+",
-
- // Attribute selectors: https://www.w3.org/TR/selectors/#attribute-selectors
- attributes = "\\[" + whitespace + "*(" + identifier + ")(?:" + whitespace +
- // Operator (capture 2)
- "*([*^$|!~]?=)" + whitespace +
- // "Attribute values must be CSS identifiers [capture 5] or strings [capture 3 or capture 4]"
- "*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|(" + identifier + "))|)" + whitespace +
- "*\\]",
-
- pseudos = ":(" + identifier + ")(?:\\((" +
- // To reduce the number of selectors needing tokenize in the preFilter, prefer arguments:
- // 1. quoted (capture 3; capture 4 or capture 5)
- "('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|" +
- // 2. simple (capture 6)
- "((?:\\\\.|[^\\\\()[\\]]|" + attributes + ")*)|" +
- // 3. anything else (capture 2)
- ".*" +
- ")\\)|)",
-
- // Leading and non-escaped trailing whitespace, capturing some non-whitespace characters preceding the latter
- rwhitespace = new RegExp( whitespace + "+", "g" ),
- rtrim = new RegExp( "^" + whitespace + "+|((?:^|[^\\\\])(?:\\\\.)*)" + whitespace + "+$", "g" ),
-
- rcomma = new RegExp( "^" + whitespace + "*," + whitespace + "*" ),
- rcombinators = new RegExp( "^" + whitespace + "*([>+~]|" + whitespace + ")" + whitespace + "*" ),
-
- rattributeQuotes = new RegExp( "=" + whitespace + "*([^\\]'\"]*?)" + whitespace + "*\\]", "g" ),
-
- rpseudo = new RegExp( pseudos ),
- ridentifier = new RegExp( "^" + identifier + "$" ),
-
- matchExpr = {
- "ID": new RegExp( "^#(" + identifier + ")" ),
- "CLASS": new RegExp( "^\\.(" + identifier + ")" ),
- "TAG": new RegExp( "^(" + identifier + "|[*])" ),
- "ATTR": new RegExp( "^" + attributes ),
- "PSEUDO": new RegExp( "^" + pseudos ),
- "CHILD": new RegExp( "^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\(" + whitespace +
- "*(even|odd|(([+-]|)(\\d*)n|)" + whitespace + "*(?:([+-]|)" + whitespace +
- "*(\\d+)|))" + whitespace + "*\\)|)", "i" ),
- "bool": new RegExp( "^(?:" + booleans + ")$", "i" ),
- // For use in libraries implementing .is()
- // We use this for POS matching in `select`
- "needsContext": new RegExp( "^" + whitespace + "*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\(" +
- whitespace + "*((?:-\\d)?\\d*)" + whitespace + "*\\)|)(?=[^-]|$)", "i" )
- },
-
- rinputs = /^(?:input|select|textarea|button)$/i,
- rheader = /^h\d$/i,
-
- rnative = /^[^{]+\{\s*\[native \w/,
-
- // Easily-parseable/retrievable ID or TAG or CLASS selectors
- rquickExpr = /^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,
-
- rsibling = /[+~]/,
-
- // CSS escapes
- // https://www.w3.org/TR/CSS21/syndata.html#escaped-characters
- runescape = new RegExp( "\\\\([\\da-f]{1,6}" + whitespace + "?|(" + whitespace + ")|.)", "ig" ),
- funescape = function( _, escaped, escapedWhitespace ) {
- var high = "0x" + escaped - 0x10000;
- // NaN means non-codepoint
- // Support: Firefox<24
- // Workaround erroneous numeric interpretation of +"0x"
- return high !== high || escapedWhitespace ?
- escaped :
- high < 0 ?
- // BMP codepoint
- String.fromCharCode( high + 0x10000 ) :
- // Supplemental Plane codepoint (surrogate pair)
- String.fromCharCode( high >> 10 | 0xD800, high & 0x3FF | 0xDC00 );
- },
-
- // CSS string/identifier serialization
- // https://drafts.csswg.org/cssom/#common-serializing-idioms
- rcssescape = /([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g,
- fcssescape = function( ch, asCodePoint ) {
- if ( asCodePoint ) {
-
- // U+0000 NULL becomes U+FFFD REPLACEMENT CHARACTER
- if ( ch === "\0" ) {
- return "\uFFFD";
- }
-
- // Control characters and (dependent upon position) numbers get escaped as code points
- return ch.slice( 0, -1 ) + "\\" + ch.charCodeAt( ch.length - 1 ).toString( 16 ) + " ";
- }
-
- // Other potentially-special ASCII characters get backslash-escaped
- return "\\" + ch;
- },
-
- // Used for iframes
- // See setDocument()
- // Removing the function wrapper causes a "Permission Denied"
- // error in IE
- unloadHandler = function() {
- setDocument();
- },
-
- disabledAncestor = addCombinator(
- function( elem ) {
- return elem.disabled === true && ("form" in elem || "label" in elem);
- },
- { dir: "parentNode", next: "legend" }
- );
-
-// Optimize for push.apply( _, NodeList )
-try {
- push.apply(
- (arr = slice.call( preferredDoc.childNodes )),
- preferredDoc.childNodes
- );
- // Support: Android<4.0
- // Detect silently failing push.apply
- arr[ preferredDoc.childNodes.length ].nodeType;
-} catch ( e ) {
- push = { apply: arr.length ?
-
- // Leverage slice if possible
- function( target, els ) {
- push_native.apply( target, slice.call(els) );
- } :
-
- // Support: IE<9
- // Otherwise append directly
- function( target, els ) {
- var j = target.length,
- i = 0;
- // Can't trust NodeList.length
- while ( (target[j++] = els[i++]) ) {}
- target.length = j - 1;
- }
- };
-}
-
-function Sizzle( selector, context, results, seed ) {
- var m, i, elem, nid, match, groups, newSelector,
- newContext = context && context.ownerDocument,
-
- // nodeType defaults to 9, since context defaults to document
- nodeType = context ? context.nodeType : 9;
-
- results = results || [];
-
- // Return early from calls with invalid selector or context
- if ( typeof selector !== "string" || !selector ||
- nodeType !== 1 && nodeType !== 9 && nodeType !== 11 ) {
-
- return results;
- }
-
- // Try to shortcut find operations (as opposed to filters) in HTML documents
- if ( !seed ) {
-
- if ( ( context ? context.ownerDocument || context : preferredDoc ) !== document ) {
- setDocument( context );
- }
- context = context || document;
-
- if ( documentIsHTML ) {
-
- // If the selector is sufficiently simple, try using a "get*By*" DOM method
- // (excepting DocumentFragment context, where the methods don't exist)
- if ( nodeType !== 11 && (match = rquickExpr.exec( selector )) ) {
-
- // ID selector
- if ( (m = match[1]) ) {
-
- // Document context
- if ( nodeType === 9 ) {
- if ( (elem = context.getElementById( m )) ) {
-
- // Support: IE, Opera, Webkit
- // TODO: identify versions
- // getElementById can match elements by name instead of ID
- if ( elem.id === m ) {
- results.push( elem );
- return results;
- }
- } else {
- return results;
- }
-
- // Element context
- } else {
-
- // Support: IE, Opera, Webkit
- // TODO: identify versions
- // getElementById can match elements by name instead of ID
- if ( newContext && (elem = newContext.getElementById( m )) &&
- contains( context, elem ) &&
- elem.id === m ) {
-
- results.push( elem );
- return results;
- }
- }
-
- // Type selector
- } else if ( match[2] ) {
- push.apply( results, context.getElementsByTagName( selector ) );
- return results;
-
- // Class selector
- } else if ( (m = match[3]) && support.getElementsByClassName &&
- context.getElementsByClassName ) {
-
- push.apply( results, context.getElementsByClassName( m ) );
- return results;
- }
- }
-
- // Take advantage of querySelectorAll
- if ( support.qsa &&
- !compilerCache[ selector + " " ] &&
- (!rbuggyQSA || !rbuggyQSA.test( selector )) ) {
-
- if ( nodeType !== 1 ) {
- newContext = context;
- newSelector = selector;
-
- // qSA looks outside Element context, which is not what we want
- // Thanks to Andrew Dupont for this workaround technique
- // Support: IE <=8
- // Exclude object elements
- } else if ( context.nodeName.toLowerCase() !== "object" ) {
-
- // Capture the context ID, setting it first if necessary
- if ( (nid = context.getAttribute( "id" )) ) {
- nid = nid.replace( rcssescape, fcssescape );
- } else {
- context.setAttribute( "id", (nid = expando) );
- }
-
- // Prefix every selector in the list
- groups = tokenize( selector );
- i = groups.length;
- while ( i-- ) {
- groups[i] = "#" + nid + " " + toSelector( groups[i] );
- }
- newSelector = groups.join( "," );
-
- // Expand context for sibling selectors
- newContext = rsibling.test( selector ) && testContext( context.parentNode ) ||
- context;
- }
-
- if ( newSelector ) {
- try {
- push.apply( results,
- newContext.querySelectorAll( newSelector )
- );
- return results;
- } catch ( qsaError ) {
- } finally {
- if ( nid === expando ) {
- context.removeAttribute( "id" );
- }
- }
- }
- }
- }
- }
-
- // All others
- return select( selector.replace( rtrim, "$1" ), context, results, seed );
-}
-
-/**
- * Create key-value caches of limited size
- * @returns {function(string, object)} Returns the Object data after storing it on itself with
- * property name the (space-suffixed) string and (if the cache is larger than Expr.cacheLength)
- * deleting the oldest entry
- */
-function createCache() {
- var keys = [];
-
- function cache( key, value ) {
- // Use (key + " ") to avoid collision with native prototype properties (see Issue #157)
- if ( keys.push( key + " " ) > Expr.cacheLength ) {
- // Only keep the most recent entries
- delete cache[ keys.shift() ];
- }
- return (cache[ key + " " ] = value);
- }
- return cache;
-}
-
-/**
- * Mark a function for special use by Sizzle
- * @param {Function} fn The function to mark
- */
-function markFunction( fn ) {
- fn[ expando ] = true;
- return fn;
-}
-
-/**
- * Support testing using an element
- * @param {Function} fn Passed the created element and returns a boolean result
- */
-function assert( fn ) {
- var el = document.createElement("fieldset");
-
- try {
- return !!fn( el );
- } catch (e) {
- return false;
- } finally {
- // Remove from its parent by default
- if ( el.parentNode ) {
- el.parentNode.removeChild( el );
- }
- // release memory in IE
- el = null;
- }
-}
-
-/**
- * Adds the same handler for all of the specified attrs
- * @param {String} attrs Pipe-separated list of attributes
- * @param {Function} handler The method that will be applied
- */
-function addHandle( attrs, handler ) {
- var arr = attrs.split("|"),
- i = arr.length;
-
- while ( i-- ) {
- Expr.attrHandle[ arr[i] ] = handler;
- }
-}
-
-/**
- * Checks document order of two siblings
- * @param {Element} a
- * @param {Element} b
- * @returns {Number} Returns less than 0 if a precedes b, greater than 0 if a follows b
- */
-function siblingCheck( a, b ) {
- var cur = b && a,
- diff = cur && a.nodeType === 1 && b.nodeType === 1 &&
- a.sourceIndex - b.sourceIndex;
-
- // Use IE sourceIndex if available on both nodes
- if ( diff ) {
- return diff;
- }
-
- // Check if b follows a
- if ( cur ) {
- while ( (cur = cur.nextSibling) ) {
- if ( cur === b ) {
- return -1;
- }
- }
- }
-
- return a ? 1 : -1;
-}
-
-/**
- * Returns a function to use in pseudos for input types
- * @param {String} type
- */
-function createInputPseudo( type ) {
- return function( elem ) {
- var name = elem.nodeName.toLowerCase();
- return name === "input" && elem.type === type;
- };
-}
-
-/**
- * Returns a function to use in pseudos for buttons
- * @param {String} type
- */
-function createButtonPseudo( type ) {
- return function( elem ) {
- var name = elem.nodeName.toLowerCase();
- return (name === "input" || name === "button") && elem.type === type;
- };
-}
-
-/**
- * Returns a function to use in pseudos for :enabled/:disabled
- * @param {Boolean} disabled true for :disabled; false for :enabled
- */
-function createDisabledPseudo( disabled ) {
-
- // Known :disabled false positives: fieldset[disabled] > legend:nth-of-type(n+2) :can-disable
- return function( elem ) {
-
- // Only certain elements can match :enabled or :disabled
- // https://html.spec.whatwg.org/multipage/scripting.html#selector-enabled
- // https://html.spec.whatwg.org/multipage/scripting.html#selector-disabled
- if ( "form" in elem ) {
-
- // Check for inherited disabledness on relevant non-disabled elements:
- // * listed form-associated elements in a disabled fieldset
- // https://html.spec.whatwg.org/multipage/forms.html#category-listed
- // https://html.spec.whatwg.org/multipage/forms.html#concept-fe-disabled
- // * option elements in a disabled optgroup
- // https://html.spec.whatwg.org/multipage/forms.html#concept-option-disabled
- // All such elements have a "form" property.
- if ( elem.parentNode && elem.disabled === false ) {
-
- // Option elements defer to a parent optgroup if present
- if ( "label" in elem ) {
- if ( "label" in elem.parentNode ) {
- return elem.parentNode.disabled === disabled;
- } else {
- return elem.disabled === disabled;
- }
- }
-
- // Support: IE 6 - 11
- // Use the isDisabled shortcut property to check for disabled fieldset ancestors
- return elem.isDisabled === disabled ||
-
- // Where there is no isDisabled, check manually
- /* jshint -W018 */
- elem.isDisabled !== !disabled &&
- disabledAncestor( elem ) === disabled;
- }
-
- return elem.disabled === disabled;
-
- // Try to winnow out elements that can't be disabled before trusting the disabled property.
- // Some victims get caught in our net (label, legend, menu, track), but it shouldn't
- // even exist on them, let alone have a boolean value.
- } else if ( "label" in elem ) {
- return elem.disabled === disabled;
- }
-
- // Remaining elements are neither :enabled nor :disabled
- return false;
- };
-}
-
-/**
- * Returns a function to use in pseudos for positionals
- * @param {Function} fn
- */
-function createPositionalPseudo( fn ) {
- return markFunction(function( argument ) {
- argument = +argument;
- return markFunction(function( seed, matches ) {
- var j,
- matchIndexes = fn( [], seed.length, argument ),
- i = matchIndexes.length;
-
- // Match elements found at the specified indexes
- while ( i-- ) {
- if ( seed[ (j = matchIndexes[i]) ] ) {
- seed[j] = !(matches[j] = seed[j]);
- }
- }
- });
- });
-}
-
-/**
- * Checks a node for validity as a Sizzle context
- * @param {Element|Object=} context
- * @returns {Element|Object|Boolean} The input node if acceptable, otherwise a falsy value
- */
-function testContext( context ) {
- return context && typeof context.getElementsByTagName !== "undefined" && context;
-}
-
-// Expose support vars for convenience
-support = Sizzle.support = {};
-
-/**
- * Detects XML nodes
- * @param {Element|Object} elem An element or a document
- * @returns {Boolean} True iff elem is a non-HTML XML node
- */
-isXML = Sizzle.isXML = function( elem ) {
- // documentElement is verified for cases where it doesn't yet exist
- // (such as loading iframes in IE - #4833)
- var documentElement = elem && (elem.ownerDocument || elem).documentElement;
- return documentElement ? documentElement.nodeName !== "HTML" : false;
-};
-
-/**
- * Sets document-related variables once based on the current document
- * @param {Element|Object} [doc] An element or document object to use to set the document
- * @returns {Object} Returns the current document
- */
-setDocument = Sizzle.setDocument = function( node ) {
- var hasCompare, subWindow,
- doc = node ? node.ownerDocument || node : preferredDoc;
-
- // Return early if doc is invalid or already selected
- if ( doc === document || doc.nodeType !== 9 || !doc.documentElement ) {
- return document;
- }
-
- // Update global variables
- document = doc;
- docElem = document.documentElement;
- documentIsHTML = !isXML( document );
-
- // Support: IE 9-11, Edge
- // Accessing iframe documents after unload throws "permission denied" errors (jQuery #13936)
- if ( preferredDoc !== document &&
- (subWindow = document.defaultView) && subWindow.top !== subWindow ) {
-
- // Support: IE 11, Edge
- if ( subWindow.addEventListener ) {
- subWindow.addEventListener( "unload", unloadHandler, false );
-
- // Support: IE 9 - 10 only
- } else if ( subWindow.attachEvent ) {
- subWindow.attachEvent( "onunload", unloadHandler );
- }
- }
-
- /* Attributes
- ---------------------------------------------------------------------- */
-
- // Support: IE<8
- // Verify that getAttribute really returns attributes and not properties
- // (excepting IE8 booleans)
- support.attributes = assert(function( el ) {
- el.className = "i";
- return !el.getAttribute("className");
- });
-
- /* getElement(s)By*
- ---------------------------------------------------------------------- */
-
- // Check if getElementsByTagName("*") returns only elements
- support.getElementsByTagName = assert(function( el ) {
- el.appendChild( document.createComment("") );
- return !el.getElementsByTagName("*").length;
- });
-
- // Support: IE<9
- support.getElementsByClassName = rnative.test( document.getElementsByClassName );
-
- // Support: IE<10
- // Check if getElementById returns elements by name
- // The broken getElementById methods don't pick up programmatically-set names,
- // so use a roundabout getElementsByName test
- support.getById = assert(function( el ) {
- docElem.appendChild( el ).id = expando;
- return !document.getElementsByName || !document.getElementsByName( expando ).length;
- });
-
- // ID filter and find
- if ( support.getById ) {
- Expr.filter["ID"] = function( id ) {
- var attrId = id.replace( runescape, funescape );
- return function( elem ) {
- return elem.getAttribute("id") === attrId;
- };
- };
- Expr.find["ID"] = function( id, context ) {
- if ( typeof context.getElementById !== "undefined" && documentIsHTML ) {
- var elem = context.getElementById( id );
- return elem ? [ elem ] : [];
- }
- };
- } else {
- Expr.filter["ID"] = function( id ) {
- var attrId = id.replace( runescape, funescape );
- return function( elem ) {
- var node = typeof elem.getAttributeNode !== "undefined" &&
- elem.getAttributeNode("id");
- return node && node.value === attrId;
- };
- };
-
- // Support: IE 6 - 7 only
- // getElementById is not reliable as a find shortcut
- Expr.find["ID"] = function( id, context ) {
- if ( typeof context.getElementById !== "undefined" && documentIsHTML ) {
- var node, i, elems,
- elem = context.getElementById( id );
-
- if ( elem ) {
-
- // Verify the id attribute
- node = elem.getAttributeNode("id");
- if ( node && node.value === id ) {
- return [ elem ];
- }
-
- // Fall back on getElementsByName
- elems = context.getElementsByName( id );
- i = 0;
- while ( (elem = elems[i++]) ) {
- node = elem.getAttributeNode("id");
- if ( node && node.value === id ) {
- return [ elem ];
- }
- }
- }
-
- return [];
- }
- };
- }
-
- // Tag
- Expr.find["TAG"] = support.getElementsByTagName ?
- function( tag, context ) {
- if ( typeof context.getElementsByTagName !== "undefined" ) {
- return context.getElementsByTagName( tag );
-
- // DocumentFragment nodes don't have gEBTN
- } else if ( support.qsa ) {
- return context.querySelectorAll( tag );
- }
- } :
-
- function( tag, context ) {
- var elem,
- tmp = [],
- i = 0,
- // By happy coincidence, a (broken) gEBTN appears on DocumentFragment nodes too
- results = context.getElementsByTagName( tag );
-
- // Filter out possible comments
- if ( tag === "*" ) {
- while ( (elem = results[i++]) ) {
- if ( elem.nodeType === 1 ) {
- tmp.push( elem );
- }
- }
-
- return tmp;
- }
- return results;
- };
-
- // Class
- Expr.find["CLASS"] = support.getElementsByClassName && function( className, context ) {
- if ( typeof context.getElementsByClassName !== "undefined" && documentIsHTML ) {
- return context.getElementsByClassName( className );
- }
- };
-
- /* QSA/matchesSelector
- ---------------------------------------------------------------------- */
-
- // QSA and matchesSelector support
-
- // matchesSelector(:active) reports false when true (IE9/Opera 11.5)
- rbuggyMatches = [];
-
- // qSa(:focus) reports false when true (Chrome 21)
- // We allow this because of a bug in IE8/9 that throws an error
- // whenever `document.activeElement` is accessed on an iframe
- // So, we allow :focus to pass through QSA all the time to avoid the IE error
- // See https://bugs.jquery.com/ticket/13378
- rbuggyQSA = [];
-
- if ( (support.qsa = rnative.test( document.querySelectorAll )) ) {
- // Build QSA regex
- // Regex strategy adopted from Diego Perini
- assert(function( el ) {
- // Select is set to empty string on purpose
- // This is to test IE's treatment of not explicitly
- // setting a boolean content attribute,
- // since its presence should be enough
- // https://bugs.jquery.com/ticket/12359
- docElem.appendChild( el ).innerHTML = "" +
- " ";
-
- // Support: IE8, Opera 11-12.16
- // Nothing should be selected when empty strings follow ^= or $= or *=
- // The test attribute must be unknown in Opera but "safe" for WinRT
- // https://msdn.microsoft.com/en-us/library/ie/hh465388.aspx#attribute_section
- if ( el.querySelectorAll("[msallowcapture^='']").length ) {
- rbuggyQSA.push( "[*^$]=" + whitespace + "*(?:''|\"\")" );
- }
-
- // Support: IE8
- // Boolean attributes and "value" are not treated correctly
- if ( !el.querySelectorAll("[selected]").length ) {
- rbuggyQSA.push( "\\[" + whitespace + "*(?:value|" + booleans + ")" );
- }
-
- // Support: Chrome<29, Android<4.4, Safari<7.0+, iOS<7.0+, PhantomJS<1.9.8+
- if ( !el.querySelectorAll( "[id~=" + expando + "-]" ).length ) {
- rbuggyQSA.push("~=");
- }
-
- // Webkit/Opera - :checked should return selected option elements
- // https://www.w3.org/TR/2011/REC-css3-selectors-20110929/#checked
- // IE8 throws error here and will not see later tests
- if ( !el.querySelectorAll(":checked").length ) {
- rbuggyQSA.push(":checked");
- }
-
- // Support: Safari 8+, iOS 8+
- // https://bugs.webkit.org/show_bug.cgi?id=136851
- // In-page `selector#id sibling-combinator selector` fails
- if ( !el.querySelectorAll( "a#" + expando + "+*" ).length ) {
- rbuggyQSA.push(".#.+[+~]");
- }
- });
-
- assert(function( el ) {
- el.innerHTML = ") matching elements by id
- for ( ; i !== len && (elem = elems[i]) != null; i++ ) {
- if ( byElement && elem ) {
- j = 0;
- if ( !context && elem.ownerDocument !== document ) {
- setDocument( elem );
- xml = !documentIsHTML;
- }
- while ( (matcher = elementMatchers[j++]) ) {
- if ( matcher( elem, context || document, xml) ) {
- results.push( elem );
- break;
- }
- }
- if ( outermost ) {
- dirruns = dirrunsUnique;
- }
- }
-
- // Track unmatched elements for set filters
- if ( bySet ) {
- // They will have gone through all possible matchers
- if ( (elem = !matcher && elem) ) {
- matchedCount--;
- }
-
- // Lengthen the array for every element, matched or not
- if ( seed ) {
- unmatched.push( elem );
- }
- }
- }
-
- // `i` is now the count of elements visited above, and adding it to `matchedCount`
- // makes the latter nonnegative.
- matchedCount += i;
-
- // Apply set filters to unmatched elements
- // NOTE: This can be skipped if there are no unmatched elements (i.e., `matchedCount`
- // equals `i`), unless we didn't visit _any_ elements in the above loop because we have
- // no element matchers and no seed.
- // Incrementing an initially-string "0" `i` allows `i` to remain a string only in that
- // case, which will result in a "00" `matchedCount` that differs from `i` but is also
- // numerically zero.
- if ( bySet && i !== matchedCount ) {
- j = 0;
- while ( (matcher = setMatchers[j++]) ) {
- matcher( unmatched, setMatched, context, xml );
- }
-
- if ( seed ) {
- // Reintegrate element matches to eliminate the need for sorting
- if ( matchedCount > 0 ) {
- while ( i-- ) {
- if ( !(unmatched[i] || setMatched[i]) ) {
- setMatched[i] = pop.call( results );
- }
- }
- }
-
- // Discard index placeholder values to get only actual matches
- setMatched = condense( setMatched );
- }
-
- // Add matches to results
- push.apply( results, setMatched );
-
- // Seedless set matches succeeding multiple successful matchers stipulate sorting
- if ( outermost && !seed && setMatched.length > 0 &&
- ( matchedCount + setMatchers.length ) > 1 ) {
-
- Sizzle.uniqueSort( results );
- }
- }
-
- // Override manipulation of globals by nested matchers
- if ( outermost ) {
- dirruns = dirrunsUnique;
- outermostContext = contextBackup;
- }
-
- return unmatched;
- };
-
- return bySet ?
- markFunction( superMatcher ) :
- superMatcher;
-}
-
-compile = Sizzle.compile = function( selector, match /* Internal Use Only */ ) {
- var i,
- setMatchers = [],
- elementMatchers = [],
- cached = compilerCache[ selector + " " ];
-
- if ( !cached ) {
- // Generate a function of recursive functions that can be used to check each element
- if ( !match ) {
- match = tokenize( selector );
- }
- i = match.length;
- while ( i-- ) {
- cached = matcherFromTokens( match[i] );
- if ( cached[ expando ] ) {
- setMatchers.push( cached );
- } else {
- elementMatchers.push( cached );
- }
- }
-
- // Cache the compiled function
- cached = compilerCache( selector, matcherFromGroupMatchers( elementMatchers, setMatchers ) );
-
- // Save selector and tokenization
- cached.selector = selector;
- }
- return cached;
-};
-
-/**
- * A low-level selection function that works with Sizzle's compiled
- * selector functions
- * @param {String|Function} selector A selector or a pre-compiled
- * selector function built with Sizzle.compile
- * @param {Element} context
- * @param {Array} [results]
- * @param {Array} [seed] A set of elements to match against
- */
-select = Sizzle.select = function( selector, context, results, seed ) {
- var i, tokens, token, type, find,
- compiled = typeof selector === "function" && selector,
- match = !seed && tokenize( (selector = compiled.selector || selector) );
-
- results = results || [];
-
- // Try to minimize operations if there is only one selector in the list and no seed
- // (the latter of which guarantees us context)
- if ( match.length === 1 ) {
-
- // Reduce context if the leading compound selector is an ID
- tokens = match[0] = match[0].slice( 0 );
- if ( tokens.length > 2 && (token = tokens[0]).type === "ID" &&
- context.nodeType === 9 && documentIsHTML && Expr.relative[ tokens[1].type ] ) {
-
- context = ( Expr.find["ID"]( token.matches[0].replace(runescape, funescape), context ) || [] )[0];
- if ( !context ) {
- return results;
-
- // Precompiled matchers will still verify ancestry, so step up a level
- } else if ( compiled ) {
- context = context.parentNode;
- }
-
- selector = selector.slice( tokens.shift().value.length );
- }
-
- // Fetch a seed set for right-to-left matching
- i = matchExpr["needsContext"].test( selector ) ? 0 : tokens.length;
- while ( i-- ) {
- token = tokens[i];
-
- // Abort if we hit a combinator
- if ( Expr.relative[ (type = token.type) ] ) {
- break;
- }
- if ( (find = Expr.find[ type ]) ) {
- // Search, expanding context for leading sibling combinators
- if ( (seed = find(
- token.matches[0].replace( runescape, funescape ),
- rsibling.test( tokens[0].type ) && testContext( context.parentNode ) || context
- )) ) {
-
- // If seed is empty or no tokens remain, we can return early
- tokens.splice( i, 1 );
- selector = seed.length && toSelector( tokens );
- if ( !selector ) {
- push.apply( results, seed );
- return results;
- }
-
- break;
- }
- }
- }
- }
-
- // Compile and execute a filtering function if one is not provided
- // Provide `match` to avoid retokenization if we modified the selector above
- ( compiled || compile( selector, match ) )(
- seed,
- context,
- !documentIsHTML,
- results,
- !context || rsibling.test( selector ) && testContext( context.parentNode ) || context
- );
- return results;
-};
-
-// One-time assignments
-
-// Sort stability
-support.sortStable = expando.split("").sort( sortOrder ).join("") === expando;
-
-// Support: Chrome 14-35+
-// Always assume duplicates if they aren't passed to the comparison function
-support.detectDuplicates = !!hasDuplicate;
-
-// Initialize against the default document
-setDocument();
-
-// Support: Webkit<537.32 - Safari 6.0.3/Chrome 25 (fixed in Chrome 27)
-// Detached nodes confoundingly follow *each other*
-support.sortDetached = assert(function( el ) {
- // Should return 1, but returns 4 (following)
- return el.compareDocumentPosition( document.createElement("fieldset") ) & 1;
-});
-
-// Support: IE<8
-// Prevent attribute/property "interpolation"
-// https://msdn.microsoft.com/en-us/library/ms536429%28VS.85%29.aspx
-if ( !assert(function( el ) {
- el.innerHTML = " to avoid XSS via location.hash (#9521)
- // Strict HTML recognition (#11290: must start with <)
- // Shortcut simple #id case for speed
- rquickExpr = /^(?:\s*(<[\w\W]+>)[^>]*|#([\w-]+))$/,
-
- init = jQuery.fn.init = function( selector, context, root ) {
- var match, elem;
-
- // HANDLE: $(""), $(null), $(undefined), $(false)
- if ( !selector ) {
- return this;
- }
-
- // Method init() accepts an alternate rootjQuery
- // so migrate can support jQuery.sub (gh-2101)
- root = root || rootjQuery;
-
- // Handle HTML strings
- if ( typeof selector === "string" ) {
- if ( selector[ 0 ] === "<" &&
- selector[ selector.length - 1 ] === ">" &&
- selector.length >= 3 ) {
-
- // Assume that strings that start and end with <> are HTML and skip the regex check
- match = [ null, selector, null ];
-
- } else {
- match = rquickExpr.exec( selector );
- }
-
- // Match html or make sure no context is specified for #id
- if ( match && ( match[ 1 ] || !context ) ) {
-
- // HANDLE: $(html) -> $(array)
- if ( match[ 1 ] ) {
- context = context instanceof jQuery ? context[ 0 ] : context;
-
- // Option to run scripts is true for back-compat
- // Intentionally let the error be thrown if parseHTML is not present
- jQuery.merge( this, jQuery.parseHTML(
- match[ 1 ],
- context && context.nodeType ? context.ownerDocument || context : document,
- true
- ) );
-
- // HANDLE: $(html, props)
- if ( rsingleTag.test( match[ 1 ] ) && jQuery.isPlainObject( context ) ) {
- for ( match in context ) {
-
- // Properties of context are called as methods if possible
- if ( jQuery.isFunction( this[ match ] ) ) {
- this[ match ]( context[ match ] );
-
- // ...and otherwise set as attributes
- } else {
- this.attr( match, context[ match ] );
- }
- }
- }
-
- return this;
-
- // HANDLE: $(#id)
- } else {
- elem = document.getElementById( match[ 2 ] );
-
- if ( elem ) {
-
- // Inject the element directly into the jQuery object
- this[ 0 ] = elem;
- this.length = 1;
- }
- return this;
- }
-
- // HANDLE: $(expr, $(...))
- } else if ( !context || context.jquery ) {
- return ( context || root ).find( selector );
-
- // HANDLE: $(expr, context)
- // (which is just equivalent to: $(context).find(expr)
- } else {
- return this.constructor( context ).find( selector );
- }
-
- // HANDLE: $(DOMElement)
- } else if ( selector.nodeType ) {
- this[ 0 ] = selector;
- this.length = 1;
- return this;
-
- // HANDLE: $(function)
- // Shortcut for document ready
- } else if ( jQuery.isFunction( selector ) ) {
- return root.ready !== undefined ?
- root.ready( selector ) :
-
- // Execute immediately if ready is not present
- selector( jQuery );
- }
-
- return jQuery.makeArray( selector, this );
- };
-
-// Give the init function the jQuery prototype for later instantiation
-init.prototype = jQuery.fn;
-
-// Initialize central reference
-rootjQuery = jQuery( document );
-
-
-var rparentsprev = /^(?:parents|prev(?:Until|All))/,
-
- // Methods guaranteed to produce a unique set when starting from a unique set
- guaranteedUnique = {
- children: true,
- contents: true,
- next: true,
- prev: true
- };
-
-jQuery.fn.extend( {
- has: function( target ) {
- var targets = jQuery( target, this ),
- l = targets.length;
-
- return this.filter( function() {
- var i = 0;
- for ( ; i < l; i++ ) {
- if ( jQuery.contains( this, targets[ i ] ) ) {
- return true;
- }
- }
- } );
- },
-
- closest: function( selectors, context ) {
- var cur,
- i = 0,
- l = this.length,
- matched = [],
- targets = typeof selectors !== "string" && jQuery( selectors );
-
- // Positional selectors never match, since there's no _selection_ context
- if ( !rneedsContext.test( selectors ) ) {
- for ( ; i < l; i++ ) {
- for ( cur = this[ i ]; cur && cur !== context; cur = cur.parentNode ) {
-
- // Always skip document fragments
- if ( cur.nodeType < 11 && ( targets ?
- targets.index( cur ) > -1 :
-
- // Don't pass non-elements to Sizzle
- cur.nodeType === 1 &&
- jQuery.find.matchesSelector( cur, selectors ) ) ) {
-
- matched.push( cur );
- break;
- }
- }
- }
- }
-
- return this.pushStack( matched.length > 1 ? jQuery.uniqueSort( matched ) : matched );
- },
-
- // Determine the position of an element within the set
- index: function( elem ) {
-
- // No argument, return index in parent
- if ( !elem ) {
- return ( this[ 0 ] && this[ 0 ].parentNode ) ? this.first().prevAll().length : -1;
- }
-
- // Index in selector
- if ( typeof elem === "string" ) {
- return indexOf.call( jQuery( elem ), this[ 0 ] );
- }
-
- // Locate the position of the desired element
- return indexOf.call( this,
-
- // If it receives a jQuery object, the first element is used
- elem.jquery ? elem[ 0 ] : elem
- );
- },
-
- add: function( selector, context ) {
- return this.pushStack(
- jQuery.uniqueSort(
- jQuery.merge( this.get(), jQuery( selector, context ) )
- )
- );
- },
-
- addBack: function( selector ) {
- return this.add( selector == null ?
- this.prevObject : this.prevObject.filter( selector )
- );
- }
-} );
-
-function sibling( cur, dir ) {
- while ( ( cur = cur[ dir ] ) && cur.nodeType !== 1 ) {}
- return cur;
-}
-
-jQuery.each( {
- parent: function( elem ) {
- var parent = elem.parentNode;
- return parent && parent.nodeType !== 11 ? parent : null;
- },
- parents: function( elem ) {
- return dir( elem, "parentNode" );
- },
- parentsUntil: function( elem, i, until ) {
- return dir( elem, "parentNode", until );
- },
- next: function( elem ) {
- return sibling( elem, "nextSibling" );
- },
- prev: function( elem ) {
- return sibling( elem, "previousSibling" );
- },
- nextAll: function( elem ) {
- return dir( elem, "nextSibling" );
- },
- prevAll: function( elem ) {
- return dir( elem, "previousSibling" );
- },
- nextUntil: function( elem, i, until ) {
- return dir( elem, "nextSibling", until );
- },
- prevUntil: function( elem, i, until ) {
- return dir( elem, "previousSibling", until );
- },
- siblings: function( elem ) {
- return siblings( ( elem.parentNode || {} ).firstChild, elem );
- },
- children: function( elem ) {
- return siblings( elem.firstChild );
- },
- contents: function( elem ) {
- if ( nodeName( elem, "iframe" ) ) {
- return elem.contentDocument;
- }
-
- // Support: IE 9 - 11 only, iOS 7 only, Android Browser <=4.3 only
- // Treat the template element as a regular one in browsers that
- // don't support it.
- if ( nodeName( elem, "template" ) ) {
- elem = elem.content || elem;
- }
-
- return jQuery.merge( [], elem.childNodes );
- }
-}, function( name, fn ) {
- jQuery.fn[ name ] = function( until, selector ) {
- var matched = jQuery.map( this, fn, until );
-
- if ( name.slice( -5 ) !== "Until" ) {
- selector = until;
- }
-
- if ( selector && typeof selector === "string" ) {
- matched = jQuery.filter( selector, matched );
- }
-
- if ( this.length > 1 ) {
-
- // Remove duplicates
- if ( !guaranteedUnique[ name ] ) {
- jQuery.uniqueSort( matched );
- }
-
- // Reverse order for parents* and prev-derivatives
- if ( rparentsprev.test( name ) ) {
- matched.reverse();
- }
- }
-
- return this.pushStack( matched );
- };
-} );
-var rnothtmlwhite = ( /[^\x20\t\r\n\f]+/g );
-
-
-
-// Convert String-formatted options into Object-formatted ones
-function createOptions( options ) {
- var object = {};
- jQuery.each( options.match( rnothtmlwhite ) || [], function( _, flag ) {
- object[ flag ] = true;
- } );
- return object;
-}
-
-/*
- * Create a callback list using the following parameters:
- *
- * options: an optional list of space-separated options that will change how
- * the callback list behaves or a more traditional option object
- *
- * By default a callback list will act like an event callback list and can be
- * "fired" multiple times.
- *
- * Possible options:
- *
- * once: will ensure the callback list can only be fired once (like a Deferred)
- *
- * memory: will keep track of previous values and will call any callback added
- * after the list has been fired right away with the latest "memorized"
- * values (like a Deferred)
- *
- * unique: will ensure a callback can only be added once (no duplicate in the list)
- *
- * stopOnFalse: interrupt callings when a callback returns false
- *
- */
-jQuery.Callbacks = function( options ) {
-
- // Convert options from String-formatted to Object-formatted if needed
- // (we check in cache first)
- options = typeof options === "string" ?
- createOptions( options ) :
- jQuery.extend( {}, options );
-
- var // Flag to know if list is currently firing
- firing,
-
- // Last fire value for non-forgettable lists
- memory,
-
- // Flag to know if list was already fired
- fired,
-
- // Flag to prevent firing
- locked,
-
- // Actual callback list
- list = [],
-
- // Queue of execution data for repeatable lists
- queue = [],
-
- // Index of currently firing callback (modified by add/remove as needed)
- firingIndex = -1,
-
- // Fire callbacks
- fire = function() {
-
- // Enforce single-firing
- locked = locked || options.once;
-
- // Execute callbacks for all pending executions,
- // respecting firingIndex overrides and runtime changes
- fired = firing = true;
- for ( ; queue.length; firingIndex = -1 ) {
- memory = queue.shift();
- while ( ++firingIndex < list.length ) {
-
- // Run callback and check for early termination
- if ( list[ firingIndex ].apply( memory[ 0 ], memory[ 1 ] ) === false &&
- options.stopOnFalse ) {
-
- // Jump to end and forget the data so .add doesn't re-fire
- firingIndex = list.length;
- memory = false;
- }
- }
- }
-
- // Forget the data if we're done with it
- if ( !options.memory ) {
- memory = false;
- }
-
- firing = false;
-
- // Clean up if we're done firing for good
- if ( locked ) {
-
- // Keep an empty list if we have data for future add calls
- if ( memory ) {
- list = [];
-
- // Otherwise, this object is spent
- } else {
- list = "";
- }
- }
- },
-
- // Actual Callbacks object
- self = {
-
- // Add a callback or a collection of callbacks to the list
- add: function() {
- if ( list ) {
-
- // If we have memory from a past run, we should fire after adding
- if ( memory && !firing ) {
- firingIndex = list.length - 1;
- queue.push( memory );
- }
-
- ( function add( args ) {
- jQuery.each( args, function( _, arg ) {
- if ( jQuery.isFunction( arg ) ) {
- if ( !options.unique || !self.has( arg ) ) {
- list.push( arg );
- }
- } else if ( arg && arg.length && jQuery.type( arg ) !== "string" ) {
-
- // Inspect recursively
- add( arg );
- }
- } );
- } )( arguments );
-
- if ( memory && !firing ) {
- fire();
- }
- }
- return this;
- },
-
- // Remove a callback from the list
- remove: function() {
- jQuery.each( arguments, function( _, arg ) {
- var index;
- while ( ( index = jQuery.inArray( arg, list, index ) ) > -1 ) {
- list.splice( index, 1 );
-
- // Handle firing indexes
- if ( index <= firingIndex ) {
- firingIndex--;
- }
- }
- } );
- return this;
- },
-
- // Check if a given callback is in the list.
- // If no argument is given, return whether or not list has callbacks attached.
- has: function( fn ) {
- return fn ?
- jQuery.inArray( fn, list ) > -1 :
- list.length > 0;
- },
-
- // Remove all callbacks from the list
- empty: function() {
- if ( list ) {
- list = [];
- }
- return this;
- },
-
- // Disable .fire and .add
- // Abort any current/pending executions
- // Clear all callbacks and values
- disable: function() {
- locked = queue = [];
- list = memory = "";
- return this;
- },
- disabled: function() {
- return !list;
- },
-
- // Disable .fire
- // Also disable .add unless we have memory (since it would have no effect)
- // Abort any pending executions
- lock: function() {
- locked = queue = [];
- if ( !memory && !firing ) {
- list = memory = "";
- }
- return this;
- },
- locked: function() {
- return !!locked;
- },
-
- // Call all callbacks with the given context and arguments
- fireWith: function( context, args ) {
- if ( !locked ) {
- args = args || [];
- args = [ context, args.slice ? args.slice() : args ];
- queue.push( args );
- if ( !firing ) {
- fire();
- }
- }
- return this;
- },
-
- // Call all the callbacks with the given arguments
- fire: function() {
- self.fireWith( this, arguments );
- return this;
- },
-
- // To know if the callbacks have already been called at least once
- fired: function() {
- return !!fired;
- }
- };
-
- return self;
-};
-
-
-function Identity( v ) {
- return v;
-}
-function Thrower( ex ) {
- throw ex;
-}
-
-function adoptValue( value, resolve, reject, noValue ) {
- var method;
-
- try {
-
- // Check for promise aspect first to privilege synchronous behavior
- if ( value && jQuery.isFunction( ( method = value.promise ) ) ) {
- method.call( value ).done( resolve ).fail( reject );
-
- // Other thenables
- } else if ( value && jQuery.isFunction( ( method = value.then ) ) ) {
- method.call( value, resolve, reject );
-
- // Other non-thenables
- } else {
-
- // Control `resolve` arguments by letting Array#slice cast boolean `noValue` to integer:
- // * false: [ value ].slice( 0 ) => resolve( value )
- // * true: [ value ].slice( 1 ) => resolve()
- resolve.apply( undefined, [ value ].slice( noValue ) );
- }
-
- // For Promises/A+, convert exceptions into rejections
- // Since jQuery.when doesn't unwrap thenables, we can skip the extra checks appearing in
- // Deferred#then to conditionally suppress rejection.
- } catch ( value ) {
-
- // Support: Android 4.0 only
- // Strict mode functions invoked without .call/.apply get global-object context
- reject.apply( undefined, [ value ] );
- }
-}
-
-jQuery.extend( {
-
- Deferred: function( func ) {
- var tuples = [
-
- // action, add listener, callbacks,
- // ... .then handlers, argument index, [final state]
- [ "notify", "progress", jQuery.Callbacks( "memory" ),
- jQuery.Callbacks( "memory" ), 2 ],
- [ "resolve", "done", jQuery.Callbacks( "once memory" ),
- jQuery.Callbacks( "once memory" ), 0, "resolved" ],
- [ "reject", "fail", jQuery.Callbacks( "once memory" ),
- jQuery.Callbacks( "once memory" ), 1, "rejected" ]
- ],
- state = "pending",
- promise = {
- state: function() {
- return state;
- },
- always: function() {
- deferred.done( arguments ).fail( arguments );
- return this;
- },
- "catch": function( fn ) {
- return promise.then( null, fn );
- },
-
- // Keep pipe for back-compat
- pipe: function( /* fnDone, fnFail, fnProgress */ ) {
- var fns = arguments;
-
- return jQuery.Deferred( function( newDefer ) {
- jQuery.each( tuples, function( i, tuple ) {
-
- // Map tuples (progress, done, fail) to arguments (done, fail, progress)
- var fn = jQuery.isFunction( fns[ tuple[ 4 ] ] ) && fns[ tuple[ 4 ] ];
-
- // deferred.progress(function() { bind to newDefer or newDefer.notify })
- // deferred.done(function() { bind to newDefer or newDefer.resolve })
- // deferred.fail(function() { bind to newDefer or newDefer.reject })
- deferred[ tuple[ 1 ] ]( function() {
- var returned = fn && fn.apply( this, arguments );
- if ( returned && jQuery.isFunction( returned.promise ) ) {
- returned.promise()
- .progress( newDefer.notify )
- .done( newDefer.resolve )
- .fail( newDefer.reject );
- } else {
- newDefer[ tuple[ 0 ] + "With" ](
- this,
- fn ? [ returned ] : arguments
- );
- }
- } );
- } );
- fns = null;
- } ).promise();
- },
- then: function( onFulfilled, onRejected, onProgress ) {
- var maxDepth = 0;
- function resolve( depth, deferred, handler, special ) {
- return function() {
- var that = this,
- args = arguments,
- mightThrow = function() {
- var returned, then;
-
- // Support: Promises/A+ section 2.3.3.3.3
- // https://promisesaplus.com/#point-59
- // Ignore double-resolution attempts
- if ( depth < maxDepth ) {
- return;
- }
-
- returned = handler.apply( that, args );
-
- // Support: Promises/A+ section 2.3.1
- // https://promisesaplus.com/#point-48
- if ( returned === deferred.promise() ) {
- throw new TypeError( "Thenable self-resolution" );
- }
-
- // Support: Promises/A+ sections 2.3.3.1, 3.5
- // https://promisesaplus.com/#point-54
- // https://promisesaplus.com/#point-75
- // Retrieve `then` only once
- then = returned &&
-
- // Support: Promises/A+ section 2.3.4
- // https://promisesaplus.com/#point-64
- // Only check objects and functions for thenability
- ( typeof returned === "object" ||
- typeof returned === "function" ) &&
- returned.then;
-
- // Handle a returned thenable
- if ( jQuery.isFunction( then ) ) {
-
- // Special processors (notify) just wait for resolution
- if ( special ) {
- then.call(
- returned,
- resolve( maxDepth, deferred, Identity, special ),
- resolve( maxDepth, deferred, Thrower, special )
- );
-
- // Normal processors (resolve) also hook into progress
- } else {
-
- // ...and disregard older resolution values
- maxDepth++;
-
- then.call(
- returned,
- resolve( maxDepth, deferred, Identity, special ),
- resolve( maxDepth, deferred, Thrower, special ),
- resolve( maxDepth, deferred, Identity,
- deferred.notifyWith )
- );
- }
-
- // Handle all other returned values
- } else {
-
- // Only substitute handlers pass on context
- // and multiple values (non-spec behavior)
- if ( handler !== Identity ) {
- that = undefined;
- args = [ returned ];
- }
-
- // Process the value(s)
- // Default process is resolve
- ( special || deferred.resolveWith )( that, args );
- }
- },
-
- // Only normal processors (resolve) catch and reject exceptions
- process = special ?
- mightThrow :
- function() {
- try {
- mightThrow();
- } catch ( e ) {
-
- if ( jQuery.Deferred.exceptionHook ) {
- jQuery.Deferred.exceptionHook( e,
- process.stackTrace );
- }
-
- // Support: Promises/A+ section 2.3.3.3.4.1
- // https://promisesaplus.com/#point-61
- // Ignore post-resolution exceptions
- if ( depth + 1 >= maxDepth ) {
-
- // Only substitute handlers pass on context
- // and multiple values (non-spec behavior)
- if ( handler !== Thrower ) {
- that = undefined;
- args = [ e ];
- }
-
- deferred.rejectWith( that, args );
- }
- }
- };
-
- // Support: Promises/A+ section 2.3.3.3.1
- // https://promisesaplus.com/#point-57
- // Re-resolve promises immediately to dodge false rejection from
- // subsequent errors
- if ( depth ) {
- process();
- } else {
-
- // Call an optional hook to record the stack, in case of exception
- // since it's otherwise lost when execution goes async
- if ( jQuery.Deferred.getStackHook ) {
- process.stackTrace = jQuery.Deferred.getStackHook();
- }
- window.setTimeout( process );
- }
- };
- }
-
- return jQuery.Deferred( function( newDefer ) {
-
- // progress_handlers.add( ... )
- tuples[ 0 ][ 3 ].add(
- resolve(
- 0,
- newDefer,
- jQuery.isFunction( onProgress ) ?
- onProgress :
- Identity,
- newDefer.notifyWith
- )
- );
-
- // fulfilled_handlers.add( ... )
- tuples[ 1 ][ 3 ].add(
- resolve(
- 0,
- newDefer,
- jQuery.isFunction( onFulfilled ) ?
- onFulfilled :
- Identity
- )
- );
-
- // rejected_handlers.add( ... )
- tuples[ 2 ][ 3 ].add(
- resolve(
- 0,
- newDefer,
- jQuery.isFunction( onRejected ) ?
- onRejected :
- Thrower
- )
- );
- } ).promise();
- },
-
- // Get a promise for this deferred
- // If obj is provided, the promise aspect is added to the object
- promise: function( obj ) {
- return obj != null ? jQuery.extend( obj, promise ) : promise;
- }
- },
- deferred = {};
-
- // Add list-specific methods
- jQuery.each( tuples, function( i, tuple ) {
- var list = tuple[ 2 ],
- stateString = tuple[ 5 ];
-
- // promise.progress = list.add
- // promise.done = list.add
- // promise.fail = list.add
- promise[ tuple[ 1 ] ] = list.add;
-
- // Handle state
- if ( stateString ) {
- list.add(
- function() {
-
- // state = "resolved" (i.e., fulfilled)
- // state = "rejected"
- state = stateString;
- },
-
- // rejected_callbacks.disable
- // fulfilled_callbacks.disable
- tuples[ 3 - i ][ 2 ].disable,
-
- // progress_callbacks.lock
- tuples[ 0 ][ 2 ].lock
- );
- }
-
- // progress_handlers.fire
- // fulfilled_handlers.fire
- // rejected_handlers.fire
- list.add( tuple[ 3 ].fire );
-
- // deferred.notify = function() { deferred.notifyWith(...) }
- // deferred.resolve = function() { deferred.resolveWith(...) }
- // deferred.reject = function() { deferred.rejectWith(...) }
- deferred[ tuple[ 0 ] ] = function() {
- deferred[ tuple[ 0 ] + "With" ]( this === deferred ? undefined : this, arguments );
- return this;
- };
-
- // deferred.notifyWith = list.fireWith
- // deferred.resolveWith = list.fireWith
- // deferred.rejectWith = list.fireWith
- deferred[ tuple[ 0 ] + "With" ] = list.fireWith;
- } );
-
- // Make the deferred a promise
- promise.promise( deferred );
-
- // Call given func if any
- if ( func ) {
- func.call( deferred, deferred );
- }
-
- // All done!
- return deferred;
- },
-
- // Deferred helper
- when: function( singleValue ) {
- var
-
- // count of uncompleted subordinates
- remaining = arguments.length,
-
- // count of unprocessed arguments
- i = remaining,
-
- // subordinate fulfillment data
- resolveContexts = Array( i ),
- resolveValues = slice.call( arguments ),
-
- // the master Deferred
- master = jQuery.Deferred(),
-
- // subordinate callback factory
- updateFunc = function( i ) {
- return function( value ) {
- resolveContexts[ i ] = this;
- resolveValues[ i ] = arguments.length > 1 ? slice.call( arguments ) : value;
- if ( !( --remaining ) ) {
- master.resolveWith( resolveContexts, resolveValues );
- }
- };
- };
-
- // Single- and empty arguments are adopted like Promise.resolve
- if ( remaining <= 1 ) {
- adoptValue( singleValue, master.done( updateFunc( i ) ).resolve, master.reject,
- !remaining );
-
- // Use .then() to unwrap secondary thenables (cf. gh-3000)
- if ( master.state() === "pending" ||
- jQuery.isFunction( resolveValues[ i ] && resolveValues[ i ].then ) ) {
-
- return master.then();
- }
- }
-
- // Multiple arguments are aggregated like Promise.all array elements
- while ( i-- ) {
- adoptValue( resolveValues[ i ], updateFunc( i ), master.reject );
- }
-
- return master.promise();
- }
-} );
-
-
-// These usually indicate a programmer mistake during development,
-// warn about them ASAP rather than swallowing them by default.
-var rerrorNames = /^(Eval|Internal|Range|Reference|Syntax|Type|URI)Error$/;
-
-jQuery.Deferred.exceptionHook = function( error, stack ) {
-
- // Support: IE 8 - 9 only
- // Console exists when dev tools are open, which can happen at any time
- if ( window.console && window.console.warn && error && rerrorNames.test( error.name ) ) {
- window.console.warn( "jQuery.Deferred exception: " + error.message, error.stack, stack );
- }
-};
-
-
-
-
-jQuery.readyException = function( error ) {
- window.setTimeout( function() {
- throw error;
- } );
-};
-
-
-
-
-// The deferred used on DOM ready
-var readyList = jQuery.Deferred();
-
-jQuery.fn.ready = function( fn ) {
-
- readyList
- .then( fn )
-
- // Wrap jQuery.readyException in a function so that the lookup
- // happens at the time of error handling instead of callback
- // registration.
- .catch( function( error ) {
- jQuery.readyException( error );
- } );
-
- return this;
-};
-
-jQuery.extend( {
-
- // Is the DOM ready to be used? Set to true once it occurs.
- isReady: false,
-
- // A counter to track how many items to wait for before
- // the ready event fires. See #6781
- readyWait: 1,
-
- // Handle when the DOM is ready
- ready: function( wait ) {
-
- // Abort if there are pending holds or we're already ready
- if ( wait === true ? --jQuery.readyWait : jQuery.isReady ) {
- return;
- }
-
- // Remember that the DOM is ready
- jQuery.isReady = true;
-
- // If a normal DOM Ready event fired, decrement, and wait if need be
- if ( wait !== true && --jQuery.readyWait > 0 ) {
- return;
- }
-
- // If there are functions bound, to execute
- readyList.resolveWith( document, [ jQuery ] );
- }
-} );
-
-jQuery.ready.then = readyList.then;
-
-// The ready event handler and self cleanup method
-function completed() {
- document.removeEventListener( "DOMContentLoaded", completed );
- window.removeEventListener( "load", completed );
- jQuery.ready();
-}
-
-// Catch cases where $(document).ready() is called
-// after the browser event has already occurred.
-// Support: IE <=9 - 10 only
-// Older IE sometimes signals "interactive" too soon
-if ( document.readyState === "complete" ||
- ( document.readyState !== "loading" && !document.documentElement.doScroll ) ) {
-
- // Handle it asynchronously to allow scripts the opportunity to delay ready
- window.setTimeout( jQuery.ready );
-
-} else {
-
- // Use the handy event callback
- document.addEventListener( "DOMContentLoaded", completed );
-
- // A fallback to window.onload, that will always work
- window.addEventListener( "load", completed );
-}
-
-
-
-
-// Multifunctional method to get and set values of a collection
-// The value/s can optionally be executed if it's a function
-var access = function( elems, fn, key, value, chainable, emptyGet, raw ) {
- var i = 0,
- len = elems.length,
- bulk = key == null;
-
- // Sets many values
- if ( jQuery.type( key ) === "object" ) {
- chainable = true;
- for ( i in key ) {
- access( elems, fn, i, key[ i ], true, emptyGet, raw );
- }
-
- // Sets one value
- } else if ( value !== undefined ) {
- chainable = true;
-
- if ( !jQuery.isFunction( value ) ) {
- raw = true;
- }
-
- if ( bulk ) {
-
- // Bulk operations run against the entire set
- if ( raw ) {
- fn.call( elems, value );
- fn = null;
-
- // ...except when executing function values
- } else {
- bulk = fn;
- fn = function( elem, key, value ) {
- return bulk.call( jQuery( elem ), value );
- };
- }
- }
-
- if ( fn ) {
- for ( ; i < len; i++ ) {
- fn(
- elems[ i ], key, raw ?
- value :
- value.call( elems[ i ], i, fn( elems[ i ], key ) )
- );
- }
- }
- }
-
- if ( chainable ) {
- return elems;
- }
-
- // Gets
- if ( bulk ) {
- return fn.call( elems );
- }
-
- return len ? fn( elems[ 0 ], key ) : emptyGet;
-};
-var acceptData = function( owner ) {
-
- // Accepts only:
- // - Node
- // - Node.ELEMENT_NODE
- // - Node.DOCUMENT_NODE
- // - Object
- // - Any
- return owner.nodeType === 1 || owner.nodeType === 9 || !( +owner.nodeType );
-};
-
-
-
-
-function Data() {
- this.expando = jQuery.expando + Data.uid++;
-}
-
-Data.uid = 1;
-
-Data.prototype = {
-
- cache: function( owner ) {
-
- // Check if the owner object already has a cache
- var value = owner[ this.expando ];
-
- // If not, create one
- if ( !value ) {
- value = {};
-
- // We can accept data for non-element nodes in modern browsers,
- // but we should not, see #8335.
- // Always return an empty object.
- if ( acceptData( owner ) ) {
-
- // If it is a node unlikely to be stringify-ed or looped over
- // use plain assignment
- if ( owner.nodeType ) {
- owner[ this.expando ] = value;
-
- // Otherwise secure it in a non-enumerable property
- // configurable must be true to allow the property to be
- // deleted when data is removed
- } else {
- Object.defineProperty( owner, this.expando, {
- value: value,
- configurable: true
- } );
- }
- }
- }
-
- return value;
- },
- set: function( owner, data, value ) {
- var prop,
- cache = this.cache( owner );
-
- // Handle: [ owner, key, value ] args
- // Always use camelCase key (gh-2257)
- if ( typeof data === "string" ) {
- cache[ jQuery.camelCase( data ) ] = value;
-
- // Handle: [ owner, { properties } ] args
- } else {
-
- // Copy the properties one-by-one to the cache object
- for ( prop in data ) {
- cache[ jQuery.camelCase( prop ) ] = data[ prop ];
- }
- }
- return cache;
- },
- get: function( owner, key ) {
- return key === undefined ?
- this.cache( owner ) :
-
- // Always use camelCase key (gh-2257)
- owner[ this.expando ] && owner[ this.expando ][ jQuery.camelCase( key ) ];
- },
- access: function( owner, key, value ) {
-
- // In cases where either:
- //
- // 1. No key was specified
- // 2. A string key was specified, but no value provided
- //
- // Take the "read" path and allow the get method to determine
- // which value to return, respectively either:
- //
- // 1. The entire cache object
- // 2. The data stored at the key
- //
- if ( key === undefined ||
- ( ( key && typeof key === "string" ) && value === undefined ) ) {
-
- return this.get( owner, key );
- }
-
- // When the key is not a string, or both a key and value
- // are specified, set or extend (existing objects) with either:
- //
- // 1. An object of properties
- // 2. A key and value
- //
- this.set( owner, key, value );
-
- // Since the "set" path can have two possible entry points
- // return the expected data based on which path was taken[*]
- return value !== undefined ? value : key;
- },
- remove: function( owner, key ) {
- var i,
- cache = owner[ this.expando ];
-
- if ( cache === undefined ) {
- return;
- }
-
- if ( key !== undefined ) {
-
- // Support array or space separated string of keys
- if ( Array.isArray( key ) ) {
-
- // If key is an array of keys...
- // We always set camelCase keys, so remove that.
- key = key.map( jQuery.camelCase );
- } else {
- key = jQuery.camelCase( key );
-
- // If a key with the spaces exists, use it.
- // Otherwise, create an array by matching non-whitespace
- key = key in cache ?
- [ key ] :
- ( key.match( rnothtmlwhite ) || [] );
- }
-
- i = key.length;
-
- while ( i-- ) {
- delete cache[ key[ i ] ];
- }
- }
-
- // Remove the expando if there's no more data
- if ( key === undefined || jQuery.isEmptyObject( cache ) ) {
-
- // Support: Chrome <=35 - 45
- // Webkit & Blink performance suffers when deleting properties
- // from DOM nodes, so set to undefined instead
- // https://bugs.chromium.org/p/chromium/issues/detail?id=378607 (bug restricted)
- if ( owner.nodeType ) {
- owner[ this.expando ] = undefined;
- } else {
- delete owner[ this.expando ];
- }
- }
- },
- hasData: function( owner ) {
- var cache = owner[ this.expando ];
- return cache !== undefined && !jQuery.isEmptyObject( cache );
- }
-};
-var dataPriv = new Data();
-
-var dataUser = new Data();
-
-
-
-// Implementation Summary
-//
-// 1. Enforce API surface and semantic compatibility with 1.9.x branch
-// 2. Improve the module's maintainability by reducing the storage
-// paths to a single mechanism.
-// 3. Use the same single mechanism to support "private" and "user" data.
-// 4. _Never_ expose "private" data to user code (TODO: Drop _data, _removeData)
-// 5. Avoid exposing implementation details on user objects (eg. expando properties)
-// 6. Provide a clear path for implementation upgrade to WeakMap in 2014
-
-var rbrace = /^(?:\{[\w\W]*\}|\[[\w\W]*\])$/,
- rmultiDash = /[A-Z]/g;
-
-function getData( data ) {
- if ( data === "true" ) {
- return true;
- }
-
- if ( data === "false" ) {
- return false;
- }
-
- if ( data === "null" ) {
- return null;
- }
-
- // Only convert to a number if it doesn't change the string
- if ( data === +data + "" ) {
- return +data;
- }
-
- if ( rbrace.test( data ) ) {
- return JSON.parse( data );
- }
-
- return data;
-}
-
-function dataAttr( elem, key, data ) {
- var name;
-
- // If nothing was found internally, try to fetch any
- // data from the HTML5 data-* attribute
- if ( data === undefined && elem.nodeType === 1 ) {
- name = "data-" + key.replace( rmultiDash, "-$&" ).toLowerCase();
- data = elem.getAttribute( name );
-
- if ( typeof data === "string" ) {
- try {
- data = getData( data );
- } catch ( e ) {}
-
- // Make sure we set the data so it isn't changed later
- dataUser.set( elem, key, data );
- } else {
- data = undefined;
- }
- }
- return data;
-}
-
-jQuery.extend( {
- hasData: function( elem ) {
- return dataUser.hasData( elem ) || dataPriv.hasData( elem );
- },
-
- data: function( elem, name, data ) {
- return dataUser.access( elem, name, data );
- },
-
- removeData: function( elem, name ) {
- dataUser.remove( elem, name );
- },
-
- // TODO: Now that all calls to _data and _removeData have been replaced
- // with direct calls to dataPriv methods, these can be deprecated.
- _data: function( elem, name, data ) {
- return dataPriv.access( elem, name, data );
- },
-
- _removeData: function( elem, name ) {
- dataPriv.remove( elem, name );
- }
-} );
-
-jQuery.fn.extend( {
- data: function( key, value ) {
- var i, name, data,
- elem = this[ 0 ],
- attrs = elem && elem.attributes;
-
- // Gets all values
- if ( key === undefined ) {
- if ( this.length ) {
- data = dataUser.get( elem );
-
- if ( elem.nodeType === 1 && !dataPriv.get( elem, "hasDataAttrs" ) ) {
- i = attrs.length;
- while ( i-- ) {
-
- // Support: IE 11 only
- // The attrs elements can be null (#14894)
- if ( attrs[ i ] ) {
- name = attrs[ i ].name;
- if ( name.indexOf( "data-" ) === 0 ) {
- name = jQuery.camelCase( name.slice( 5 ) );
- dataAttr( elem, name, data[ name ] );
- }
- }
- }
- dataPriv.set( elem, "hasDataAttrs", true );
- }
- }
-
- return data;
- }
-
- // Sets multiple values
- if ( typeof key === "object" ) {
- return this.each( function() {
- dataUser.set( this, key );
- } );
- }
-
- return access( this, function( value ) {
- var data;
-
- // The calling jQuery object (element matches) is not empty
- // (and therefore has an element appears at this[ 0 ]) and the
- // `value` parameter was not undefined. An empty jQuery object
- // will result in `undefined` for elem = this[ 0 ] which will
- // throw an exception if an attempt to read a data cache is made.
- if ( elem && value === undefined ) {
-
- // Attempt to get data from the cache
- // The key will always be camelCased in Data
- data = dataUser.get( elem, key );
- if ( data !== undefined ) {
- return data;
- }
-
- // Attempt to "discover" the data in
- // HTML5 custom data-* attrs
- data = dataAttr( elem, key );
- if ( data !== undefined ) {
- return data;
- }
-
- // We tried really hard, but the data doesn't exist.
- return;
- }
-
- // Set the data...
- this.each( function() {
-
- // We always store the camelCased key
- dataUser.set( this, key, value );
- } );
- }, null, value, arguments.length > 1, null, true );
- },
-
- removeData: function( key ) {
- return this.each( function() {
- dataUser.remove( this, key );
- } );
- }
-} );
-
-
-jQuery.extend( {
- queue: function( elem, type, data ) {
- var queue;
-
- if ( elem ) {
- type = ( type || "fx" ) + "queue";
- queue = dataPriv.get( elem, type );
-
- // Speed up dequeue by getting out quickly if this is just a lookup
- if ( data ) {
- if ( !queue || Array.isArray( data ) ) {
- queue = dataPriv.access( elem, type, jQuery.makeArray( data ) );
- } else {
- queue.push( data );
- }
- }
- return queue || [];
- }
- },
-
- dequeue: function( elem, type ) {
- type = type || "fx";
-
- var queue = jQuery.queue( elem, type ),
- startLength = queue.length,
- fn = queue.shift(),
- hooks = jQuery._queueHooks( elem, type ),
- next = function() {
- jQuery.dequeue( elem, type );
- };
-
- // If the fx queue is dequeued, always remove the progress sentinel
- if ( fn === "inprogress" ) {
- fn = queue.shift();
- startLength--;
- }
-
- if ( fn ) {
-
- // Add a progress sentinel to prevent the fx queue from being
- // automatically dequeued
- if ( type === "fx" ) {
- queue.unshift( "inprogress" );
- }
-
- // Clear up the last queue stop function
- delete hooks.stop;
- fn.call( elem, next, hooks );
- }
-
- if ( !startLength && hooks ) {
- hooks.empty.fire();
- }
- },
-
- // Not public - generate a queueHooks object, or return the current one
- _queueHooks: function( elem, type ) {
- var key = type + "queueHooks";
- return dataPriv.get( elem, key ) || dataPriv.access( elem, key, {
- empty: jQuery.Callbacks( "once memory" ).add( function() {
- dataPriv.remove( elem, [ type + "queue", key ] );
- } )
- } );
- }
-} );
-
-jQuery.fn.extend( {
- queue: function( type, data ) {
- var setter = 2;
-
- if ( typeof type !== "string" ) {
- data = type;
- type = "fx";
- setter--;
- }
-
- if ( arguments.length < setter ) {
- return jQuery.queue( this[ 0 ], type );
- }
-
- return data === undefined ?
- this :
- this.each( function() {
- var queue = jQuery.queue( this, type, data );
-
- // Ensure a hooks for this queue
- jQuery._queueHooks( this, type );
-
- if ( type === "fx" && queue[ 0 ] !== "inprogress" ) {
- jQuery.dequeue( this, type );
- }
- } );
- },
- dequeue: function( type ) {
- return this.each( function() {
- jQuery.dequeue( this, type );
- } );
- },
- clearQueue: function( type ) {
- return this.queue( type || "fx", [] );
- },
-
- // Get a promise resolved when queues of a certain type
- // are emptied (fx is the type by default)
- promise: function( type, obj ) {
- var tmp,
- count = 1,
- defer = jQuery.Deferred(),
- elements = this,
- i = this.length,
- resolve = function() {
- if ( !( --count ) ) {
- defer.resolveWith( elements, [ elements ] );
- }
- };
-
- if ( typeof type !== "string" ) {
- obj = type;
- type = undefined;
- }
- type = type || "fx";
-
- while ( i-- ) {
- tmp = dataPriv.get( elements[ i ], type + "queueHooks" );
- if ( tmp && tmp.empty ) {
- count++;
- tmp.empty.add( resolve );
- }
- }
- resolve();
- return defer.promise( obj );
- }
-} );
-var pnum = ( /[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/ ).source;
-
-var rcssNum = new RegExp( "^(?:([+-])=|)(" + pnum + ")([a-z%]*)$", "i" );
-
-
-var cssExpand = [ "Top", "Right", "Bottom", "Left" ];
-
-var isHiddenWithinTree = function( elem, el ) {
-
- // isHiddenWithinTree might be called from jQuery#filter function;
- // in that case, element will be second argument
- elem = el || elem;
-
- // Inline style trumps all
- return elem.style.display === "none" ||
- elem.style.display === "" &&
-
- // Otherwise, check computed style
- // Support: Firefox <=43 - 45
- // Disconnected elements can have computed display: none, so first confirm that elem is
- // in the document.
- jQuery.contains( elem.ownerDocument, elem ) &&
-
- jQuery.css( elem, "display" ) === "none";
- };
-
-var swap = function( elem, options, callback, args ) {
- var ret, name,
- old = {};
-
- // Remember the old values, and insert the new ones
- for ( name in options ) {
- old[ name ] = elem.style[ name ];
- elem.style[ name ] = options[ name ];
- }
-
- ret = callback.apply( elem, args || [] );
-
- // Revert the old values
- for ( name in options ) {
- elem.style[ name ] = old[ name ];
- }
-
- return ret;
-};
-
-
-
-
-function adjustCSS( elem, prop, valueParts, tween ) {
- var adjusted,
- scale = 1,
- maxIterations = 20,
- currentValue = tween ?
- function() {
- return tween.cur();
- } :
- function() {
- return jQuery.css( elem, prop, "" );
- },
- initial = currentValue(),
- unit = valueParts && valueParts[ 3 ] || ( jQuery.cssNumber[ prop ] ? "" : "px" ),
-
- // Starting value computation is required for potential unit mismatches
- initialInUnit = ( jQuery.cssNumber[ prop ] || unit !== "px" && +initial ) &&
- rcssNum.exec( jQuery.css( elem, prop ) );
-
- if ( initialInUnit && initialInUnit[ 3 ] !== unit ) {
-
- // Trust units reported by jQuery.css
- unit = unit || initialInUnit[ 3 ];
-
- // Make sure we update the tween properties later on
- valueParts = valueParts || [];
-
- // Iteratively approximate from a nonzero starting point
- initialInUnit = +initial || 1;
-
- do {
-
- // If previous iteration zeroed out, double until we get *something*.
- // Use string for doubling so we don't accidentally see scale as unchanged below
- scale = scale || ".5";
-
- // Adjust and apply
- initialInUnit = initialInUnit / scale;
- jQuery.style( elem, prop, initialInUnit + unit );
-
- // Update scale, tolerating zero or NaN from tween.cur()
- // Break the loop if scale is unchanged or perfect, or if we've just had enough.
- } while (
- scale !== ( scale = currentValue() / initial ) && scale !== 1 && --maxIterations
- );
- }
-
- if ( valueParts ) {
- initialInUnit = +initialInUnit || +initial || 0;
-
- // Apply relative offset (+=/-=) if specified
- adjusted = valueParts[ 1 ] ?
- initialInUnit + ( valueParts[ 1 ] + 1 ) * valueParts[ 2 ] :
- +valueParts[ 2 ];
- if ( tween ) {
- tween.unit = unit;
- tween.start = initialInUnit;
- tween.end = adjusted;
- }
- }
- return adjusted;
-}
-
-
-var defaultDisplayMap = {};
-
-function getDefaultDisplay( elem ) {
- var temp,
- doc = elem.ownerDocument,
- nodeName = elem.nodeName,
- display = defaultDisplayMap[ nodeName ];
-
- if ( display ) {
- return display;
- }
-
- temp = doc.body.appendChild( doc.createElement( nodeName ) );
- display = jQuery.css( temp, "display" );
-
- temp.parentNode.removeChild( temp );
-
- if ( display === "none" ) {
- display = "block";
- }
- defaultDisplayMap[ nodeName ] = display;
-
- return display;
-}
-
-function showHide( elements, show ) {
- var display, elem,
- values = [],
- index = 0,
- length = elements.length;
-
- // Determine new display value for elements that need to change
- for ( ; index < length; index++ ) {
- elem = elements[ index ];
- if ( !elem.style ) {
- continue;
- }
-
- display = elem.style.display;
- if ( show ) {
-
- // Since we force visibility upon cascade-hidden elements, an immediate (and slow)
- // check is required in this first loop unless we have a nonempty display value (either
- // inline or about-to-be-restored)
- if ( display === "none" ) {
- values[ index ] = dataPriv.get( elem, "display" ) || null;
- if ( !values[ index ] ) {
- elem.style.display = "";
- }
- }
- if ( elem.style.display === "" && isHiddenWithinTree( elem ) ) {
- values[ index ] = getDefaultDisplay( elem );
- }
- } else {
- if ( display !== "none" ) {
- values[ index ] = "none";
-
- // Remember what we're overwriting
- dataPriv.set( elem, "display", display );
- }
- }
- }
-
- // Set the display of the elements in a second loop to avoid constant reflow
- for ( index = 0; index < length; index++ ) {
- if ( values[ index ] != null ) {
- elements[ index ].style.display = values[ index ];
- }
- }
-
- return elements;
-}
-
-jQuery.fn.extend( {
- show: function() {
- return showHide( this, true );
- },
- hide: function() {
- return showHide( this );
- },
- toggle: function( state ) {
- if ( typeof state === "boolean" ) {
- return state ? this.show() : this.hide();
- }
-
- return this.each( function() {
- if ( isHiddenWithinTree( this ) ) {
- jQuery( this ).show();
- } else {
- jQuery( this ).hide();
- }
- } );
- }
-} );
-var rcheckableType = ( /^(?:checkbox|radio)$/i );
-
-var rtagName = ( /<([a-z][^\/\0>\x20\t\r\n\f]+)/i );
-
-var rscriptType = ( /^$|\/(?:java|ecma)script/i );
-
-
-
-// We have to close these tags to support XHTML (#13200)
-var wrapMap = {
-
- // Support: IE <=9 only
- option: [ 1, "", " " ],
-
- // XHTML parsers do not magically insert elements in the
- // same way that tag soup parsers do. So we cannot shorten
- // this by omitting or other required elements.
- thead: [ 1, "" ],
- col: [ 2, "" ],
- tr: [ 2, "" ],
- td: [ 3, "" ],
-
- _default: [ 0, "", "" ]
-};
-
-// Support: IE <=9 only
-wrapMap.optgroup = wrapMap.option;
-
-wrapMap.tbody = wrapMap.tfoot = wrapMap.colgroup = wrapMap.caption = wrapMap.thead;
-wrapMap.th = wrapMap.td;
-
-
-function getAll( context, tag ) {
-
- // Support: IE <=9 - 11 only
- // Use typeof to avoid zero-argument method invocation on host objects (#15151)
- var ret;
-
- if ( typeof context.getElementsByTagName !== "undefined" ) {
- ret = context.getElementsByTagName( tag || "*" );
-
- } else if ( typeof context.querySelectorAll !== "undefined" ) {
- ret = context.querySelectorAll( tag || "*" );
-
- } else {
- ret = [];
- }
-
- if ( tag === undefined || tag && nodeName( context, tag ) ) {
- return jQuery.merge( [ context ], ret );
- }
-
- return ret;
-}
-
-
-// Mark scripts as having already been evaluated
-function setGlobalEval( elems, refElements ) {
- var i = 0,
- l = elems.length;
-
- for ( ; i < l; i++ ) {
- dataPriv.set(
- elems[ i ],
- "globalEval",
- !refElements || dataPriv.get( refElements[ i ], "globalEval" )
- );
- }
-}
-
-
-var rhtml = /<|&#?\w+;/;
-
-function buildFragment( elems, context, scripts, selection, ignored ) {
- var elem, tmp, tag, wrap, contains, j,
- fragment = context.createDocumentFragment(),
- nodes = [],
- i = 0,
- l = elems.length;
-
- for ( ; i < l; i++ ) {
- elem = elems[ i ];
-
- if ( elem || elem === 0 ) {
-
- // Add nodes directly
- if ( jQuery.type( elem ) === "object" ) {
-
- // Support: Android <=4.0 only, PhantomJS 1 only
- // push.apply(_, arraylike) throws on ancient WebKit
- jQuery.merge( nodes, elem.nodeType ? [ elem ] : elem );
-
- // Convert non-html into a text node
- } else if ( !rhtml.test( elem ) ) {
- nodes.push( context.createTextNode( elem ) );
-
- // Convert html into DOM nodes
- } else {
- tmp = tmp || fragment.appendChild( context.createElement( "div" ) );
-
- // Deserialize a standard representation
- tag = ( rtagName.exec( elem ) || [ "", "" ] )[ 1 ].toLowerCase();
- wrap = wrapMap[ tag ] || wrapMap._default;
- tmp.innerHTML = wrap[ 1 ] + jQuery.htmlPrefilter( elem ) + wrap[ 2 ];
-
- // Descend through wrappers to the right content
- j = wrap[ 0 ];
- while ( j-- ) {
- tmp = tmp.lastChild;
- }
-
- // Support: Android <=4.0 only, PhantomJS 1 only
- // push.apply(_, arraylike) throws on ancient WebKit
- jQuery.merge( nodes, tmp.childNodes );
-
- // Remember the top-level container
- tmp = fragment.firstChild;
-
- // Ensure the created nodes are orphaned (#12392)
- tmp.textContent = "";
- }
- }
- }
-
- // Remove wrapper from fragment
- fragment.textContent = "";
-
- i = 0;
- while ( ( elem = nodes[ i++ ] ) ) {
-
- // Skip elements already in the context collection (trac-4087)
- if ( selection && jQuery.inArray( elem, selection ) > -1 ) {
- if ( ignored ) {
- ignored.push( elem );
- }
- continue;
- }
-
- contains = jQuery.contains( elem.ownerDocument, elem );
-
- // Append to fragment
- tmp = getAll( fragment.appendChild( elem ), "script" );
-
- // Preserve script evaluation history
- if ( contains ) {
- setGlobalEval( tmp );
- }
-
- // Capture executables
- if ( scripts ) {
- j = 0;
- while ( ( elem = tmp[ j++ ] ) ) {
- if ( rscriptType.test( elem.type || "" ) ) {
- scripts.push( elem );
- }
- }
- }
- }
-
- return fragment;
-}
-
-
-( function() {
- var fragment = document.createDocumentFragment(),
- div = fragment.appendChild( document.createElement( "div" ) ),
- input = document.createElement( "input" );
-
- // Support: Android 4.0 - 4.3 only
- // Check state lost if the name is set (#11217)
- // Support: Windows Web Apps (WWA)
- // `name` and `type` must use .setAttribute for WWA (#14901)
- input.setAttribute( "type", "radio" );
- input.setAttribute( "checked", "checked" );
- input.setAttribute( "name", "t" );
-
- div.appendChild( input );
-
- // Support: Android <=4.1 only
- // Older WebKit doesn't clone checked state correctly in fragments
- support.checkClone = div.cloneNode( true ).cloneNode( true ).lastChild.checked;
-
- // Support: IE <=11 only
- // Make sure textarea (and checkbox) defaultValue is properly cloned
- div.innerHTML = "";
- support.noCloneChecked = !!div.cloneNode( true ).lastChild.defaultValue;
-} )();
-var documentElement = document.documentElement;
-
-
-
-var
- rkeyEvent = /^key/,
- rmouseEvent = /^(?:mouse|pointer|contextmenu|drag|drop)|click/,
- rtypenamespace = /^([^.]*)(?:\.(.+)|)/;
-
-function returnTrue() {
- return true;
-}
-
-function returnFalse() {
- return false;
-}
-
-// Support: IE <=9 only
-// See #13393 for more info
-function safeActiveElement() {
- try {
- return document.activeElement;
- } catch ( err ) { }
-}
-
-function on( elem, types, selector, data, fn, one ) {
- var origFn, type;
-
- // Types can be a map of types/handlers
- if ( typeof types === "object" ) {
-
- // ( types-Object, selector, data )
- if ( typeof selector !== "string" ) {
-
- // ( types-Object, data )
- data = data || selector;
- selector = undefined;
- }
- for ( type in types ) {
- on( elem, type, selector, data, types[ type ], one );
- }
- return elem;
- }
-
- if ( data == null && fn == null ) {
-
- // ( types, fn )
- fn = selector;
- data = selector = undefined;
- } else if ( fn == null ) {
- if ( typeof selector === "string" ) {
-
- // ( types, selector, fn )
- fn = data;
- data = undefined;
- } else {
-
- // ( types, data, fn )
- fn = data;
- data = selector;
- selector = undefined;
- }
- }
- if ( fn === false ) {
- fn = returnFalse;
- } else if ( !fn ) {
- return elem;
- }
-
- if ( one === 1 ) {
- origFn = fn;
- fn = function( event ) {
-
- // Can use an empty set, since event contains the info
- jQuery().off( event );
- return origFn.apply( this, arguments );
- };
-
- // Use same guid so caller can remove using origFn
- fn.guid = origFn.guid || ( origFn.guid = jQuery.guid++ );
- }
- return elem.each( function() {
- jQuery.event.add( this, types, fn, data, selector );
- } );
-}
-
-/*
- * Helper functions for managing events -- not part of the public interface.
- * Props to Dean Edwards' addEvent library for many of the ideas.
- */
-jQuery.event = {
-
- global: {},
-
- add: function( elem, types, handler, data, selector ) {
-
- var handleObjIn, eventHandle, tmp,
- events, t, handleObj,
- special, handlers, type, namespaces, origType,
- elemData = dataPriv.get( elem );
-
- // Don't attach events to noData or text/comment nodes (but allow plain objects)
- if ( !elemData ) {
- return;
- }
-
- // Caller can pass in an object of custom data in lieu of the handler
- if ( handler.handler ) {
- handleObjIn = handler;
- handler = handleObjIn.handler;
- selector = handleObjIn.selector;
- }
-
- // Ensure that invalid selectors throw exceptions at attach time
- // Evaluate against documentElement in case elem is a non-element node (e.g., document)
- if ( selector ) {
- jQuery.find.matchesSelector( documentElement, selector );
- }
-
- // Make sure that the handler has a unique ID, used to find/remove it later
- if ( !handler.guid ) {
- handler.guid = jQuery.guid++;
- }
-
- // Init the element's event structure and main handler, if this is the first
- if ( !( events = elemData.events ) ) {
- events = elemData.events = {};
- }
- if ( !( eventHandle = elemData.handle ) ) {
- eventHandle = elemData.handle = function( e ) {
-
- // Discard the second event of a jQuery.event.trigger() and
- // when an event is called after a page has unloaded
- return typeof jQuery !== "undefined" && jQuery.event.triggered !== e.type ?
- jQuery.event.dispatch.apply( elem, arguments ) : undefined;
- };
- }
-
- // Handle multiple events separated by a space
- types = ( types || "" ).match( rnothtmlwhite ) || [ "" ];
- t = types.length;
- while ( t-- ) {
- tmp = rtypenamespace.exec( types[ t ] ) || [];
- type = origType = tmp[ 1 ];
- namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort();
-
- // There *must* be a type, no attaching namespace-only handlers
- if ( !type ) {
- continue;
- }
-
- // If event changes its type, use the special event handlers for the changed type
- special = jQuery.event.special[ type ] || {};
-
- // If selector defined, determine special event api type, otherwise given type
- type = ( selector ? special.delegateType : special.bindType ) || type;
-
- // Update special based on newly reset type
- special = jQuery.event.special[ type ] || {};
-
- // handleObj is passed to all event handlers
- handleObj = jQuery.extend( {
- type: type,
- origType: origType,
- data: data,
- handler: handler,
- guid: handler.guid,
- selector: selector,
- needsContext: selector && jQuery.expr.match.needsContext.test( selector ),
- namespace: namespaces.join( "." )
- }, handleObjIn );
-
- // Init the event handler queue if we're the first
- if ( !( handlers = events[ type ] ) ) {
- handlers = events[ type ] = [];
- handlers.delegateCount = 0;
-
- // Only use addEventListener if the special events handler returns false
- if ( !special.setup ||
- special.setup.call( elem, data, namespaces, eventHandle ) === false ) {
-
- if ( elem.addEventListener ) {
- elem.addEventListener( type, eventHandle );
- }
- }
- }
-
- if ( special.add ) {
- special.add.call( elem, handleObj );
-
- if ( !handleObj.handler.guid ) {
- handleObj.handler.guid = handler.guid;
- }
- }
-
- // Add to the element's handler list, delegates in front
- if ( selector ) {
- handlers.splice( handlers.delegateCount++, 0, handleObj );
- } else {
- handlers.push( handleObj );
- }
-
- // Keep track of which events have ever been used, for event optimization
- jQuery.event.global[ type ] = true;
- }
-
- },
-
- // Detach an event or set of events from an element
- remove: function( elem, types, handler, selector, mappedTypes ) {
-
- var j, origCount, tmp,
- events, t, handleObj,
- special, handlers, type, namespaces, origType,
- elemData = dataPriv.hasData( elem ) && dataPriv.get( elem );
-
- if ( !elemData || !( events = elemData.events ) ) {
- return;
- }
-
- // Once for each type.namespace in types; type may be omitted
- types = ( types || "" ).match( rnothtmlwhite ) || [ "" ];
- t = types.length;
- while ( t-- ) {
- tmp = rtypenamespace.exec( types[ t ] ) || [];
- type = origType = tmp[ 1 ];
- namespaces = ( tmp[ 2 ] || "" ).split( "." ).sort();
-
- // Unbind all events (on this namespace, if provided) for the element
- if ( !type ) {
- for ( type in events ) {
- jQuery.event.remove( elem, type + types[ t ], handler, selector, true );
- }
- continue;
- }
-
- special = jQuery.event.special[ type ] || {};
- type = ( selector ? special.delegateType : special.bindType ) || type;
- handlers = events[ type ] || [];
- tmp = tmp[ 2 ] &&
- new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" );
-
- // Remove matching events
- origCount = j = handlers.length;
- while ( j-- ) {
- handleObj = handlers[ j ];
-
- if ( ( mappedTypes || origType === handleObj.origType ) &&
- ( !handler || handler.guid === handleObj.guid ) &&
- ( !tmp || tmp.test( handleObj.namespace ) ) &&
- ( !selector || selector === handleObj.selector ||
- selector === "**" && handleObj.selector ) ) {
- handlers.splice( j, 1 );
-
- if ( handleObj.selector ) {
- handlers.delegateCount--;
- }
- if ( special.remove ) {
- special.remove.call( elem, handleObj );
- }
- }
- }
-
- // Remove generic event handler if we removed something and no more handlers exist
- // (avoids potential for endless recursion during removal of special event handlers)
- if ( origCount && !handlers.length ) {
- if ( !special.teardown ||
- special.teardown.call( elem, namespaces, elemData.handle ) === false ) {
-
- jQuery.removeEvent( elem, type, elemData.handle );
- }
-
- delete events[ type ];
- }
- }
-
- // Remove data and the expando if it's no longer used
- if ( jQuery.isEmptyObject( events ) ) {
- dataPriv.remove( elem, "handle events" );
- }
- },
-
- dispatch: function( nativeEvent ) {
-
- // Make a writable jQuery.Event from the native event object
- var event = jQuery.event.fix( nativeEvent );
-
- var i, j, ret, matched, handleObj, handlerQueue,
- args = new Array( arguments.length ),
- handlers = ( dataPriv.get( this, "events" ) || {} )[ event.type ] || [],
- special = jQuery.event.special[ event.type ] || {};
-
- // Use the fix-ed jQuery.Event rather than the (read-only) native event
- args[ 0 ] = event;
-
- for ( i = 1; i < arguments.length; i++ ) {
- args[ i ] = arguments[ i ];
- }
-
- event.delegateTarget = this;
-
- // Call the preDispatch hook for the mapped type, and let it bail if desired
- if ( special.preDispatch && special.preDispatch.call( this, event ) === false ) {
- return;
- }
-
- // Determine handlers
- handlerQueue = jQuery.event.handlers.call( this, event, handlers );
-
- // Run delegates first; they may want to stop propagation beneath us
- i = 0;
- while ( ( matched = handlerQueue[ i++ ] ) && !event.isPropagationStopped() ) {
- event.currentTarget = matched.elem;
-
- j = 0;
- while ( ( handleObj = matched.handlers[ j++ ] ) &&
- !event.isImmediatePropagationStopped() ) {
-
- // Triggered event must either 1) have no namespace, or 2) have namespace(s)
- // a subset or equal to those in the bound event (both can have no namespace).
- if ( !event.rnamespace || event.rnamespace.test( handleObj.namespace ) ) {
-
- event.handleObj = handleObj;
- event.data = handleObj.data;
-
- ret = ( ( jQuery.event.special[ handleObj.origType ] || {} ).handle ||
- handleObj.handler ).apply( matched.elem, args );
-
- if ( ret !== undefined ) {
- if ( ( event.result = ret ) === false ) {
- event.preventDefault();
- event.stopPropagation();
- }
- }
- }
- }
- }
-
- // Call the postDispatch hook for the mapped type
- if ( special.postDispatch ) {
- special.postDispatch.call( this, event );
- }
-
- return event.result;
- },
-
- handlers: function( event, handlers ) {
- var i, handleObj, sel, matchedHandlers, matchedSelectors,
- handlerQueue = [],
- delegateCount = handlers.delegateCount,
- cur = event.target;
-
- // Find delegate handlers
- if ( delegateCount &&
-
- // Support: IE <=9
- // Black-hole SVG instance trees (trac-13180)
- cur.nodeType &&
-
- // Support: Firefox <=42
- // Suppress spec-violating clicks indicating a non-primary pointer button (trac-3861)
- // https://www.w3.org/TR/DOM-Level-3-Events/#event-type-click
- // Support: IE 11 only
- // ...but not arrow key "clicks" of radio inputs, which can have `button` -1 (gh-2343)
- !( event.type === "click" && event.button >= 1 ) ) {
-
- for ( ; cur !== this; cur = cur.parentNode || this ) {
-
- // Don't check non-elements (#13208)
- // Don't process clicks on disabled elements (#6911, #8165, #11382, #11764)
- if ( cur.nodeType === 1 && !( event.type === "click" && cur.disabled === true ) ) {
- matchedHandlers = [];
- matchedSelectors = {};
- for ( i = 0; i < delegateCount; i++ ) {
- handleObj = handlers[ i ];
-
- // Don't conflict with Object.prototype properties (#13203)
- sel = handleObj.selector + " ";
-
- if ( matchedSelectors[ sel ] === undefined ) {
- matchedSelectors[ sel ] = handleObj.needsContext ?
- jQuery( sel, this ).index( cur ) > -1 :
- jQuery.find( sel, this, null, [ cur ] ).length;
- }
- if ( matchedSelectors[ sel ] ) {
- matchedHandlers.push( handleObj );
- }
- }
- if ( matchedHandlers.length ) {
- handlerQueue.push( { elem: cur, handlers: matchedHandlers } );
- }
- }
- }
- }
-
- // Add the remaining (directly-bound) handlers
- cur = this;
- if ( delegateCount < handlers.length ) {
- handlerQueue.push( { elem: cur, handlers: handlers.slice( delegateCount ) } );
- }
-
- return handlerQueue;
- },
-
- addProp: function( name, hook ) {
- Object.defineProperty( jQuery.Event.prototype, name, {
- enumerable: true,
- configurable: true,
-
- get: jQuery.isFunction( hook ) ?
- function() {
- if ( this.originalEvent ) {
- return hook( this.originalEvent );
- }
- } :
- function() {
- if ( this.originalEvent ) {
- return this.originalEvent[ name ];
- }
- },
-
- set: function( value ) {
- Object.defineProperty( this, name, {
- enumerable: true,
- configurable: true,
- writable: true,
- value: value
- } );
- }
- } );
- },
-
- fix: function( originalEvent ) {
- return originalEvent[ jQuery.expando ] ?
- originalEvent :
- new jQuery.Event( originalEvent );
- },
-
- special: {
- load: {
-
- // Prevent triggered image.load events from bubbling to window.load
- noBubble: true
- },
- focus: {
-
- // Fire native event if possible so blur/focus sequence is correct
- trigger: function() {
- if ( this !== safeActiveElement() && this.focus ) {
- this.focus();
- return false;
- }
- },
- delegateType: "focusin"
- },
- blur: {
- trigger: function() {
- if ( this === safeActiveElement() && this.blur ) {
- this.blur();
- return false;
- }
- },
- delegateType: "focusout"
- },
- click: {
-
- // For checkbox, fire native event so checked state will be right
- trigger: function() {
- if ( this.type === "checkbox" && this.click && nodeName( this, "input" ) ) {
- this.click();
- return false;
- }
- },
-
- // For cross-browser consistency, don't fire native .click() on links
- _default: function( event ) {
- return nodeName( event.target, "a" );
- }
- },
-
- beforeunload: {
- postDispatch: function( event ) {
-
- // Support: Firefox 20+
- // Firefox doesn't alert if the returnValue field is not set.
- if ( event.result !== undefined && event.originalEvent ) {
- event.originalEvent.returnValue = event.result;
- }
- }
- }
- }
-};
-
-jQuery.removeEvent = function( elem, type, handle ) {
-
- // This "if" is needed for plain objects
- if ( elem.removeEventListener ) {
- elem.removeEventListener( type, handle );
- }
-};
-
-jQuery.Event = function( src, props ) {
-
- // Allow instantiation without the 'new' keyword
- if ( !( this instanceof jQuery.Event ) ) {
- return new jQuery.Event( src, props );
- }
-
- // Event object
- if ( src && src.type ) {
- this.originalEvent = src;
- this.type = src.type;
-
- // Events bubbling up the document may have been marked as prevented
- // by a handler lower down the tree; reflect the correct value.
- this.isDefaultPrevented = src.defaultPrevented ||
- src.defaultPrevented === undefined &&
-
- // Support: Android <=2.3 only
- src.returnValue === false ?
- returnTrue :
- returnFalse;
-
- // Create target properties
- // Support: Safari <=6 - 7 only
- // Target should not be a text node (#504, #13143)
- this.target = ( src.target && src.target.nodeType === 3 ) ?
- src.target.parentNode :
- src.target;
-
- this.currentTarget = src.currentTarget;
- this.relatedTarget = src.relatedTarget;
-
- // Event type
- } else {
- this.type = src;
- }
-
- // Put explicitly provided properties onto the event object
- if ( props ) {
- jQuery.extend( this, props );
- }
-
- // Create a timestamp if incoming event doesn't have one
- this.timeStamp = src && src.timeStamp || jQuery.now();
-
- // Mark it as fixed
- this[ jQuery.expando ] = true;
-};
-
-// jQuery.Event is based on DOM3 Events as specified by the ECMAScript Language Binding
-// https://www.w3.org/TR/2003/WD-DOM-Level-3-Events-20030331/ecma-script-binding.html
-jQuery.Event.prototype = {
- constructor: jQuery.Event,
- isDefaultPrevented: returnFalse,
- isPropagationStopped: returnFalse,
- isImmediatePropagationStopped: returnFalse,
- isSimulated: false,
-
- preventDefault: function() {
- var e = this.originalEvent;
-
- this.isDefaultPrevented = returnTrue;
-
- if ( e && !this.isSimulated ) {
- e.preventDefault();
- }
- },
- stopPropagation: function() {
- var e = this.originalEvent;
-
- this.isPropagationStopped = returnTrue;
-
- if ( e && !this.isSimulated ) {
- e.stopPropagation();
- }
- },
- stopImmediatePropagation: function() {
- var e = this.originalEvent;
-
- this.isImmediatePropagationStopped = returnTrue;
-
- if ( e && !this.isSimulated ) {
- e.stopImmediatePropagation();
- }
-
- this.stopPropagation();
- }
-};
-
-// Includes all common event props including KeyEvent and MouseEvent specific props
-jQuery.each( {
- altKey: true,
- bubbles: true,
- cancelable: true,
- changedTouches: true,
- ctrlKey: true,
- detail: true,
- eventPhase: true,
- metaKey: true,
- pageX: true,
- pageY: true,
- shiftKey: true,
- view: true,
- "char": true,
- charCode: true,
- key: true,
- keyCode: true,
- button: true,
- buttons: true,
- clientX: true,
- clientY: true,
- offsetX: true,
- offsetY: true,
- pointerId: true,
- pointerType: true,
- screenX: true,
- screenY: true,
- targetTouches: true,
- toElement: true,
- touches: true,
-
- which: function( event ) {
- var button = event.button;
-
- // Add which for key events
- if ( event.which == null && rkeyEvent.test( event.type ) ) {
- return event.charCode != null ? event.charCode : event.keyCode;
- }
-
- // Add which for click: 1 === left; 2 === middle; 3 === right
- if ( !event.which && button !== undefined && rmouseEvent.test( event.type ) ) {
- if ( button & 1 ) {
- return 1;
- }
-
- if ( button & 2 ) {
- return 3;
- }
-
- if ( button & 4 ) {
- return 2;
- }
-
- return 0;
- }
-
- return event.which;
- }
-}, jQuery.event.addProp );
-
-// Create mouseenter/leave events using mouseover/out and event-time checks
-// so that event delegation works in jQuery.
-// Do the same for pointerenter/pointerleave and pointerover/pointerout
-//
-// Support: Safari 7 only
-// Safari sends mouseenter too often; see:
-// https://bugs.chromium.org/p/chromium/issues/detail?id=470258
-// for the description of the bug (it existed in older Chrome versions as well).
-jQuery.each( {
- mouseenter: "mouseover",
- mouseleave: "mouseout",
- pointerenter: "pointerover",
- pointerleave: "pointerout"
-}, function( orig, fix ) {
- jQuery.event.special[ orig ] = {
- delegateType: fix,
- bindType: fix,
-
- handle: function( event ) {
- var ret,
- target = this,
- related = event.relatedTarget,
- handleObj = event.handleObj;
-
- // For mouseenter/leave call the handler if related is outside the target.
- // NB: No relatedTarget if the mouse left/entered the browser window
- if ( !related || ( related !== target && !jQuery.contains( target, related ) ) ) {
- event.type = handleObj.origType;
- ret = handleObj.handler.apply( this, arguments );
- event.type = fix;
- }
- return ret;
- }
- };
-} );
-
-jQuery.fn.extend( {
-
- on: function( types, selector, data, fn ) {
- return on( this, types, selector, data, fn );
- },
- one: function( types, selector, data, fn ) {
- return on( this, types, selector, data, fn, 1 );
- },
- off: function( types, selector, fn ) {
- var handleObj, type;
- if ( types && types.preventDefault && types.handleObj ) {
-
- // ( event ) dispatched jQuery.Event
- handleObj = types.handleObj;
- jQuery( types.delegateTarget ).off(
- handleObj.namespace ?
- handleObj.origType + "." + handleObj.namespace :
- handleObj.origType,
- handleObj.selector,
- handleObj.handler
- );
- return this;
- }
- if ( typeof types === "object" ) {
-
- // ( types-object [, selector] )
- for ( type in types ) {
- this.off( type, selector, types[ type ] );
- }
- return this;
- }
- if ( selector === false || typeof selector === "function" ) {
-
- // ( types [, fn] )
- fn = selector;
- selector = undefined;
- }
- if ( fn === false ) {
- fn = returnFalse;
- }
- return this.each( function() {
- jQuery.event.remove( this, types, fn, selector );
- } );
- }
-} );
-
-
-var
-
- /* eslint-disable max-len */
-
- // See https://github.com/eslint/eslint/issues/3229
- rxhtmlTag = /<(?!area|br|col|embed|hr|img|input|link|meta|param)(([a-z][^\/\0>\x20\t\r\n\f]*)[^>]*)\/>/gi,
-
- /* eslint-enable */
-
- // Support: IE <=10 - 11, Edge 12 - 13
- // In IE/Edge using regex groups here causes severe slowdowns.
- // See https://connect.microsoft.com/IE/feedback/details/1736512/
- rnoInnerhtml = /\s*$/g;
-
-// Prefer a tbody over its parent table for containing new rows
-function manipulationTarget( elem, content ) {
- if ( nodeName( elem, "table" ) &&
- nodeName( content.nodeType !== 11 ? content : content.firstChild, "tr" ) ) {
-
- return jQuery( ">tbody", elem )[ 0 ] || elem;
- }
-
- return elem;
-}
-
-// Replace/restore the type attribute of script elements for safe DOM manipulation
-function disableScript( elem ) {
- elem.type = ( elem.getAttribute( "type" ) !== null ) + "/" + elem.type;
- return elem;
-}
-function restoreScript( elem ) {
- var match = rscriptTypeMasked.exec( elem.type );
-
- if ( match ) {
- elem.type = match[ 1 ];
- } else {
- elem.removeAttribute( "type" );
- }
-
- return elem;
-}
-
-function cloneCopyEvent( src, dest ) {
- var i, l, type, pdataOld, pdataCur, udataOld, udataCur, events;
-
- if ( dest.nodeType !== 1 ) {
- return;
- }
-
- // 1. Copy private data: events, handlers, etc.
- if ( dataPriv.hasData( src ) ) {
- pdataOld = dataPriv.access( src );
- pdataCur = dataPriv.set( dest, pdataOld );
- events = pdataOld.events;
-
- if ( events ) {
- delete pdataCur.handle;
- pdataCur.events = {};
-
- for ( type in events ) {
- for ( i = 0, l = events[ type ].length; i < l; i++ ) {
- jQuery.event.add( dest, type, events[ type ][ i ] );
- }
- }
- }
- }
-
- // 2. Copy user data
- if ( dataUser.hasData( src ) ) {
- udataOld = dataUser.access( src );
- udataCur = jQuery.extend( {}, udataOld );
-
- dataUser.set( dest, udataCur );
- }
-}
-
-// Fix IE bugs, see support tests
-function fixInput( src, dest ) {
- var nodeName = dest.nodeName.toLowerCase();
-
- // Fails to persist the checked state of a cloned checkbox or radio button.
- if ( nodeName === "input" && rcheckableType.test( src.type ) ) {
- dest.checked = src.checked;
-
- // Fails to return the selected option to the default selected state when cloning options
- } else if ( nodeName === "input" || nodeName === "textarea" ) {
- dest.defaultValue = src.defaultValue;
- }
-}
-
-function domManip( collection, args, callback, ignored ) {
-
- // Flatten any nested arrays
- args = concat.apply( [], args );
-
- var fragment, first, scripts, hasScripts, node, doc,
- i = 0,
- l = collection.length,
- iNoClone = l - 1,
- value = args[ 0 ],
- isFunction = jQuery.isFunction( value );
-
- // We can't cloneNode fragments that contain checked, in WebKit
- if ( isFunction ||
- ( l > 1 && typeof value === "string" &&
- !support.checkClone && rchecked.test( value ) ) ) {
- return collection.each( function( index ) {
- var self = collection.eq( index );
- if ( isFunction ) {
- args[ 0 ] = value.call( this, index, self.html() );
- }
- domManip( self, args, callback, ignored );
- } );
- }
-
- if ( l ) {
- fragment = buildFragment( args, collection[ 0 ].ownerDocument, false, collection, ignored );
- first = fragment.firstChild;
-
- if ( fragment.childNodes.length === 1 ) {
- fragment = first;
- }
-
- // Require either new content or an interest in ignored elements to invoke the callback
- if ( first || ignored ) {
- scripts = jQuery.map( getAll( fragment, "script" ), disableScript );
- hasScripts = scripts.length;
-
- // Use the original fragment for the last item
- // instead of the first because it can end up
- // being emptied incorrectly in certain situations (#8070).
- for ( ; i < l; i++ ) {
- node = fragment;
-
- if ( i !== iNoClone ) {
- node = jQuery.clone( node, true, true );
-
- // Keep references to cloned scripts for later restoration
- if ( hasScripts ) {
-
- // Support: Android <=4.0 only, PhantomJS 1 only
- // push.apply(_, arraylike) throws on ancient WebKit
- jQuery.merge( scripts, getAll( node, "script" ) );
- }
- }
-
- callback.call( collection[ i ], node, i );
- }
-
- if ( hasScripts ) {
- doc = scripts[ scripts.length - 1 ].ownerDocument;
-
- // Reenable scripts
- jQuery.map( scripts, restoreScript );
-
- // Evaluate executable scripts on first document insertion
- for ( i = 0; i < hasScripts; i++ ) {
- node = scripts[ i ];
- if ( rscriptType.test( node.type || "" ) &&
- !dataPriv.access( node, "globalEval" ) &&
- jQuery.contains( doc, node ) ) {
-
- if ( node.src ) {
-
- // Optional AJAX dependency, but won't run scripts if not present
- if ( jQuery._evalUrl ) {
- jQuery._evalUrl( node.src );
- }
- } else {
- DOMEval( node.textContent.replace( rcleanScript, "" ), doc );
- }
- }
- }
- }
- }
- }
-
- return collection;
-}
-
-function remove( elem, selector, keepData ) {
- var node,
- nodes = selector ? jQuery.filter( selector, elem ) : elem,
- i = 0;
-
- for ( ; ( node = nodes[ i ] ) != null; i++ ) {
- if ( !keepData && node.nodeType === 1 ) {
- jQuery.cleanData( getAll( node ) );
- }
-
- if ( node.parentNode ) {
- if ( keepData && jQuery.contains( node.ownerDocument, node ) ) {
- setGlobalEval( getAll( node, "script" ) );
- }
- node.parentNode.removeChild( node );
- }
- }
-
- return elem;
-}
-
-jQuery.extend( {
- htmlPrefilter: function( html ) {
- return html.replace( rxhtmlTag, "<$1>" );
- },
-
- clone: function( elem, dataAndEvents, deepDataAndEvents ) {
- var i, l, srcElements, destElements,
- clone = elem.cloneNode( true ),
- inPage = jQuery.contains( elem.ownerDocument, elem );
-
- // Fix IE cloning issues
- if ( !support.noCloneChecked && ( elem.nodeType === 1 || elem.nodeType === 11 ) &&
- !jQuery.isXMLDoc( elem ) ) {
-
- // We eschew Sizzle here for performance reasons: https://jsperf.com/getall-vs-sizzle/2
- destElements = getAll( clone );
- srcElements = getAll( elem );
-
- for ( i = 0, l = srcElements.length; i < l; i++ ) {
- fixInput( srcElements[ i ], destElements[ i ] );
- }
- }
-
- // Copy the events from the original to the clone
- if ( dataAndEvents ) {
- if ( deepDataAndEvents ) {
- srcElements = srcElements || getAll( elem );
- destElements = destElements || getAll( clone );
-
- for ( i = 0, l = srcElements.length; i < l; i++ ) {
- cloneCopyEvent( srcElements[ i ], destElements[ i ] );
- }
- } else {
- cloneCopyEvent( elem, clone );
- }
- }
-
- // Preserve script evaluation history
- destElements = getAll( clone, "script" );
- if ( destElements.length > 0 ) {
- setGlobalEval( destElements, !inPage && getAll( elem, "script" ) );
- }
-
- // Return the cloned set
- return clone;
- },
-
- cleanData: function( elems ) {
- var data, elem, type,
- special = jQuery.event.special,
- i = 0;
-
- for ( ; ( elem = elems[ i ] ) !== undefined; i++ ) {
- if ( acceptData( elem ) ) {
- if ( ( data = elem[ dataPriv.expando ] ) ) {
- if ( data.events ) {
- for ( type in data.events ) {
- if ( special[ type ] ) {
- jQuery.event.remove( elem, type );
-
- // This is a shortcut to avoid jQuery.event.remove's overhead
- } else {
- jQuery.removeEvent( elem, type, data.handle );
- }
- }
- }
-
- // Support: Chrome <=35 - 45+
- // Assign undefined instead of using delete, see Data#remove
- elem[ dataPriv.expando ] = undefined;
- }
- if ( elem[ dataUser.expando ] ) {
-
- // Support: Chrome <=35 - 45+
- // Assign undefined instead of using delete, see Data#remove
- elem[ dataUser.expando ] = undefined;
- }
- }
- }
- }
-} );
-
-jQuery.fn.extend( {
- detach: function( selector ) {
- return remove( this, selector, true );
- },
-
- remove: function( selector ) {
- return remove( this, selector );
- },
-
- text: function( value ) {
- return access( this, function( value ) {
- return value === undefined ?
- jQuery.text( this ) :
- this.empty().each( function() {
- if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) {
- this.textContent = value;
- }
- } );
- }, null, value, arguments.length );
- },
-
- append: function() {
- return domManip( this, arguments, function( elem ) {
- if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) {
- var target = manipulationTarget( this, elem );
- target.appendChild( elem );
- }
- } );
- },
-
- prepend: function() {
- return domManip( this, arguments, function( elem ) {
- if ( this.nodeType === 1 || this.nodeType === 11 || this.nodeType === 9 ) {
- var target = manipulationTarget( this, elem );
- target.insertBefore( elem, target.firstChild );
- }
- } );
- },
-
- before: function() {
- return domManip( this, arguments, function( elem ) {
- if ( this.parentNode ) {
- this.parentNode.insertBefore( elem, this );
- }
- } );
- },
-
- after: function() {
- return domManip( this, arguments, function( elem ) {
- if ( this.parentNode ) {
- this.parentNode.insertBefore( elem, this.nextSibling );
- }
- } );
- },
-
- empty: function() {
- var elem,
- i = 0;
-
- for ( ; ( elem = this[ i ] ) != null; i++ ) {
- if ( elem.nodeType === 1 ) {
-
- // Prevent memory leaks
- jQuery.cleanData( getAll( elem, false ) );
-
- // Remove any remaining nodes
- elem.textContent = "";
- }
- }
-
- return this;
- },
-
- clone: function( dataAndEvents, deepDataAndEvents ) {
- dataAndEvents = dataAndEvents == null ? false : dataAndEvents;
- deepDataAndEvents = deepDataAndEvents == null ? dataAndEvents : deepDataAndEvents;
-
- return this.map( function() {
- return jQuery.clone( this, dataAndEvents, deepDataAndEvents );
- } );
- },
-
- html: function( value ) {
- return access( this, function( value ) {
- var elem = this[ 0 ] || {},
- i = 0,
- l = this.length;
-
- if ( value === undefined && elem.nodeType === 1 ) {
- return elem.innerHTML;
- }
-
- // See if we can take a shortcut and just use innerHTML
- if ( typeof value === "string" && !rnoInnerhtml.test( value ) &&
- !wrapMap[ ( rtagName.exec( value ) || [ "", "" ] )[ 1 ].toLowerCase() ] ) {
-
- value = jQuery.htmlPrefilter( value );
-
- try {
- for ( ; i < l; i++ ) {
- elem = this[ i ] || {};
-
- // Remove element nodes and prevent memory leaks
- if ( elem.nodeType === 1 ) {
- jQuery.cleanData( getAll( elem, false ) );
- elem.innerHTML = value;
- }
- }
-
- elem = 0;
-
- // If using innerHTML throws an exception, use the fallback method
- } catch ( e ) {}
- }
-
- if ( elem ) {
- this.empty().append( value );
- }
- }, null, value, arguments.length );
- },
-
- replaceWith: function() {
- var ignored = [];
-
- // Make the changes, replacing each non-ignored context element with the new content
- return domManip( this, arguments, function( elem ) {
- var parent = this.parentNode;
-
- if ( jQuery.inArray( this, ignored ) < 0 ) {
- jQuery.cleanData( getAll( this ) );
- if ( parent ) {
- parent.replaceChild( elem, this );
- }
- }
-
- // Force callback invocation
- }, ignored );
- }
-} );
-
-jQuery.each( {
- appendTo: "append",
- prependTo: "prepend",
- insertBefore: "before",
- insertAfter: "after",
- replaceAll: "replaceWith"
-}, function( name, original ) {
- jQuery.fn[ name ] = function( selector ) {
- var elems,
- ret = [],
- insert = jQuery( selector ),
- last = insert.length - 1,
- i = 0;
-
- for ( ; i <= last; i++ ) {
- elems = i === last ? this : this.clone( true );
- jQuery( insert[ i ] )[ original ]( elems );
-
- // Support: Android <=4.0 only, PhantomJS 1 only
- // .get() because push.apply(_, arraylike) throws on ancient WebKit
- push.apply( ret, elems.get() );
- }
-
- return this.pushStack( ret );
- };
-} );
-var rmargin = ( /^margin/ );
-
-var rnumnonpx = new RegExp( "^(" + pnum + ")(?!px)[a-z%]+$", "i" );
-
-var getStyles = function( elem ) {
-
- // Support: IE <=11 only, Firefox <=30 (#15098, #14150)
- // IE throws on elements created in popups
- // FF meanwhile throws on frame elements through "defaultView.getComputedStyle"
- var view = elem.ownerDocument.defaultView;
-
- if ( !view || !view.opener ) {
- view = window;
- }
-
- return view.getComputedStyle( elem );
- };
-
-
-
-( function() {
-
- // Executing both pixelPosition & boxSizingReliable tests require only one layout
- // so they're executed at the same time to save the second computation.
- function computeStyleTests() {
-
- // This is a singleton, we need to execute it only once
- if ( !div ) {
- return;
- }
-
- div.style.cssText =
- "box-sizing:border-box;" +
- "position:relative;display:block;" +
- "margin:auto;border:1px;padding:1px;" +
- "top:1%;width:50%";
- div.innerHTML = "";
- documentElement.appendChild( container );
-
- var divStyle = window.getComputedStyle( div );
- pixelPositionVal = divStyle.top !== "1%";
-
- // Support: Android 4.0 - 4.3 only, Firefox <=3 - 44
- reliableMarginLeftVal = divStyle.marginLeft === "2px";
- boxSizingReliableVal = divStyle.width === "4px";
-
- // Support: Android 4.0 - 4.3 only
- // Some styles come back with percentage values, even though they shouldn't
- div.style.marginRight = "50%";
- pixelMarginRightVal = divStyle.marginRight === "4px";
-
- documentElement.removeChild( container );
-
- // Nullify the div so it wouldn't be stored in the memory and
- // it will also be a sign that checks already performed
- div = null;
- }
-
- var pixelPositionVal, boxSizingReliableVal, pixelMarginRightVal, reliableMarginLeftVal,
- container = document.createElement( "div" ),
- div = document.createElement( "div" );
-
- // Finish early in limited (non-browser) environments
- if ( !div.style ) {
- return;
- }
-
- // Support: IE <=9 - 11 only
- // Style of cloned element affects source element cloned (#8908)
- div.style.backgroundClip = "content-box";
- div.cloneNode( true ).style.backgroundClip = "";
- support.clearCloneStyle = div.style.backgroundClip === "content-box";
-
- container.style.cssText = "border:0;width:8px;height:0;top:0;left:-9999px;" +
- "padding:0;margin-top:1px;position:absolute";
- container.appendChild( div );
-
- jQuery.extend( support, {
- pixelPosition: function() {
- computeStyleTests();
- return pixelPositionVal;
- },
- boxSizingReliable: function() {
- computeStyleTests();
- return boxSizingReliableVal;
- },
- pixelMarginRight: function() {
- computeStyleTests();
- return pixelMarginRightVal;
- },
- reliableMarginLeft: function() {
- computeStyleTests();
- return reliableMarginLeftVal;
- }
- } );
-} )();
-
-
-function curCSS( elem, name, computed ) {
- var width, minWidth, maxWidth, ret,
-
- // Support: Firefox 51+
- // Retrieving style before computed somehow
- // fixes an issue with getting wrong values
- // on detached elements
- style = elem.style;
-
- computed = computed || getStyles( elem );
-
- // getPropertyValue is needed for:
- // .css('filter') (IE 9 only, #12537)
- // .css('--customProperty) (#3144)
- if ( computed ) {
- ret = computed.getPropertyValue( name ) || computed[ name ];
-
- if ( ret === "" && !jQuery.contains( elem.ownerDocument, elem ) ) {
- ret = jQuery.style( elem, name );
- }
-
- // A tribute to the "awesome hack by Dean Edwards"
- // Android Browser returns percentage for some values,
- // but width seems to be reliably pixels.
- // This is against the CSSOM draft spec:
- // https://drafts.csswg.org/cssom/#resolved-values
- if ( !support.pixelMarginRight() && rnumnonpx.test( ret ) && rmargin.test( name ) ) {
-
- // Remember the original values
- width = style.width;
- minWidth = style.minWidth;
- maxWidth = style.maxWidth;
-
- // Put in the new values to get a computed value out
- style.minWidth = style.maxWidth = style.width = ret;
- ret = computed.width;
-
- // Revert the changed values
- style.width = width;
- style.minWidth = minWidth;
- style.maxWidth = maxWidth;
- }
- }
-
- return ret !== undefined ?
-
- // Support: IE <=9 - 11 only
- // IE returns zIndex value as an integer.
- ret + "" :
- ret;
-}
-
-
-function addGetHookIf( conditionFn, hookFn ) {
-
- // Define the hook, we'll check on the first run if it's really needed.
- return {
- get: function() {
- if ( conditionFn() ) {
-
- // Hook not needed (or it's not possible to use it due
- // to missing dependency), remove it.
- delete this.get;
- return;
- }
-
- // Hook needed; redefine it so that the support test is not executed again.
- return ( this.get = hookFn ).apply( this, arguments );
- }
- };
-}
-
-
-var
-
- // Swappable if display is none or starts with table
- // except "table", "table-cell", or "table-caption"
- // See here for display values: https://developer.mozilla.org/en-US/docs/CSS/display
- rdisplayswap = /^(none|table(?!-c[ea]).+)/,
- rcustomProp = /^--/,
- cssShow = { position: "absolute", visibility: "hidden", display: "block" },
- cssNormalTransform = {
- letterSpacing: "0",
- fontWeight: "400"
- },
-
- cssPrefixes = [ "Webkit", "Moz", "ms" ],
- emptyStyle = document.createElement( "div" ).style;
-
-// Return a css property mapped to a potentially vendor prefixed property
-function vendorPropName( name ) {
-
- // Shortcut for names that are not vendor prefixed
- if ( name in emptyStyle ) {
- return name;
- }
-
- // Check for vendor prefixed names
- var capName = name[ 0 ].toUpperCase() + name.slice( 1 ),
- i = cssPrefixes.length;
-
- while ( i-- ) {
- name = cssPrefixes[ i ] + capName;
- if ( name in emptyStyle ) {
- return name;
- }
- }
-}
-
-// Return a property mapped along what jQuery.cssProps suggests or to
-// a vendor prefixed property.
-function finalPropName( name ) {
- var ret = jQuery.cssProps[ name ];
- if ( !ret ) {
- ret = jQuery.cssProps[ name ] = vendorPropName( name ) || name;
- }
- return ret;
-}
-
-function setPositiveNumber( elem, value, subtract ) {
-
- // Any relative (+/-) values have already been
- // normalized at this point
- var matches = rcssNum.exec( value );
- return matches ?
-
- // Guard against undefined "subtract", e.g., when used as in cssHooks
- Math.max( 0, matches[ 2 ] - ( subtract || 0 ) ) + ( matches[ 3 ] || "px" ) :
- value;
-}
-
-function augmentWidthOrHeight( elem, name, extra, isBorderBox, styles ) {
- var i,
- val = 0;
-
- // If we already have the right measurement, avoid augmentation
- if ( extra === ( isBorderBox ? "border" : "content" ) ) {
- i = 4;
-
- // Otherwise initialize for horizontal or vertical properties
- } else {
- i = name === "width" ? 1 : 0;
- }
-
- for ( ; i < 4; i += 2 ) {
-
- // Both box models exclude margin, so add it if we want it
- if ( extra === "margin" ) {
- val += jQuery.css( elem, extra + cssExpand[ i ], true, styles );
- }
-
- if ( isBorderBox ) {
-
- // border-box includes padding, so remove it if we want content
- if ( extra === "content" ) {
- val -= jQuery.css( elem, "padding" + cssExpand[ i ], true, styles );
- }
-
- // At this point, extra isn't border nor margin, so remove border
- if ( extra !== "margin" ) {
- val -= jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles );
- }
- } else {
-
- // At this point, extra isn't content, so add padding
- val += jQuery.css( elem, "padding" + cssExpand[ i ], true, styles );
-
- // At this point, extra isn't content nor padding, so add border
- if ( extra !== "padding" ) {
- val += jQuery.css( elem, "border" + cssExpand[ i ] + "Width", true, styles );
- }
- }
- }
-
- return val;
-}
-
-function getWidthOrHeight( elem, name, extra ) {
-
- // Start with computed style
- var valueIsBorderBox,
- styles = getStyles( elem ),
- val = curCSS( elem, name, styles ),
- isBorderBox = jQuery.css( elem, "boxSizing", false, styles ) === "border-box";
-
- // Computed unit is not pixels. Stop here and return.
- if ( rnumnonpx.test( val ) ) {
- return val;
- }
-
- // Check for style in case a browser which returns unreliable values
- // for getComputedStyle silently falls back to the reliable elem.style
- valueIsBorderBox = isBorderBox &&
- ( support.boxSizingReliable() || val === elem.style[ name ] );
-
- // Fall back to offsetWidth/Height when value is "auto"
- // This happens for inline elements with no explicit setting (gh-3571)
- if ( val === "auto" ) {
- val = elem[ "offset" + name[ 0 ].toUpperCase() + name.slice( 1 ) ];
- }
-
- // Normalize "", auto, and prepare for extra
- val = parseFloat( val ) || 0;
-
- // Use the active box-sizing model to add/subtract irrelevant styles
- return ( val +
- augmentWidthOrHeight(
- elem,
- name,
- extra || ( isBorderBox ? "border" : "content" ),
- valueIsBorderBox,
- styles
- )
- ) + "px";
-}
-
-jQuery.extend( {
-
- // Add in style property hooks for overriding the default
- // behavior of getting and setting a style property
- cssHooks: {
- opacity: {
- get: function( elem, computed ) {
- if ( computed ) {
-
- // We should always get a number back from opacity
- var ret = curCSS( elem, "opacity" );
- return ret === "" ? "1" : ret;
- }
- }
- }
- },
-
- // Don't automatically add "px" to these possibly-unitless properties
- cssNumber: {
- "animationIterationCount": true,
- "columnCount": true,
- "fillOpacity": true,
- "flexGrow": true,
- "flexShrink": true,
- "fontWeight": true,
- "lineHeight": true,
- "opacity": true,
- "order": true,
- "orphans": true,
- "widows": true,
- "zIndex": true,
- "zoom": true
- },
-
- // Add in properties whose names you wish to fix before
- // setting or getting the value
- cssProps: {
- "float": "cssFloat"
- },
-
- // Get and set the style property on a DOM Node
- style: function( elem, name, value, extra ) {
-
- // Don't set styles on text and comment nodes
- if ( !elem || elem.nodeType === 3 || elem.nodeType === 8 || !elem.style ) {
- return;
- }
-
- // Make sure that we're working with the right name
- var ret, type, hooks,
- origName = jQuery.camelCase( name ),
- isCustomProp = rcustomProp.test( name ),
- style = elem.style;
-
- // Make sure that we're working with the right name. We don't
- // want to query the value if it is a CSS custom property
- // since they are user-defined.
- if ( !isCustomProp ) {
- name = finalPropName( origName );
- }
-
- // Gets hook for the prefixed version, then unprefixed version
- hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ];
-
- // Check if we're setting a value
- if ( value !== undefined ) {
- type = typeof value;
-
- // Convert "+=" or "-=" to relative numbers (#7345)
- if ( type === "string" && ( ret = rcssNum.exec( value ) ) && ret[ 1 ] ) {
- value = adjustCSS( elem, name, ret );
-
- // Fixes bug #9237
- type = "number";
- }
-
- // Make sure that null and NaN values aren't set (#7116)
- if ( value == null || value !== value ) {
- return;
- }
-
- // If a number was passed in, add the unit (except for certain CSS properties)
- if ( type === "number" ) {
- value += ret && ret[ 3 ] || ( jQuery.cssNumber[ origName ] ? "" : "px" );
- }
-
- // background-* props affect original clone's values
- if ( !support.clearCloneStyle && value === "" && name.indexOf( "background" ) === 0 ) {
- style[ name ] = "inherit";
- }
-
- // If a hook was provided, use that value, otherwise just set the specified value
- if ( !hooks || !( "set" in hooks ) ||
- ( value = hooks.set( elem, value, extra ) ) !== undefined ) {
-
- if ( isCustomProp ) {
- style.setProperty( name, value );
- } else {
- style[ name ] = value;
- }
- }
-
- } else {
-
- // If a hook was provided get the non-computed value from there
- if ( hooks && "get" in hooks &&
- ( ret = hooks.get( elem, false, extra ) ) !== undefined ) {
-
- return ret;
- }
-
- // Otherwise just get the value from the style object
- return style[ name ];
- }
- },
-
- css: function( elem, name, extra, styles ) {
- var val, num, hooks,
- origName = jQuery.camelCase( name ),
- isCustomProp = rcustomProp.test( name );
-
- // Make sure that we're working with the right name. We don't
- // want to modify the value if it is a CSS custom property
- // since they are user-defined.
- if ( !isCustomProp ) {
- name = finalPropName( origName );
- }
-
- // Try prefixed name followed by the unprefixed name
- hooks = jQuery.cssHooks[ name ] || jQuery.cssHooks[ origName ];
-
- // If a hook was provided get the computed value from there
- if ( hooks && "get" in hooks ) {
- val = hooks.get( elem, true, extra );
- }
-
- // Otherwise, if a way to get the computed value exists, use that
- if ( val === undefined ) {
- val = curCSS( elem, name, styles );
- }
-
- // Convert "normal" to computed value
- if ( val === "normal" && name in cssNormalTransform ) {
- val = cssNormalTransform[ name ];
- }
-
- // Make numeric if forced or a qualifier was provided and val looks numeric
- if ( extra === "" || extra ) {
- num = parseFloat( val );
- return extra === true || isFinite( num ) ? num || 0 : val;
- }
-
- return val;
- }
-} );
-
-jQuery.each( [ "height", "width" ], function( i, name ) {
- jQuery.cssHooks[ name ] = {
- get: function( elem, computed, extra ) {
- if ( computed ) {
-
- // Certain elements can have dimension info if we invisibly show them
- // but it must have a current display style that would benefit
- return rdisplayswap.test( jQuery.css( elem, "display" ) ) &&
-
- // Support: Safari 8+
- // Table columns in Safari have non-zero offsetWidth & zero
- // getBoundingClientRect().width unless display is changed.
- // Support: IE <=11 only
- // Running getBoundingClientRect on a disconnected node
- // in IE throws an error.
- ( !elem.getClientRects().length || !elem.getBoundingClientRect().width ) ?
- swap( elem, cssShow, function() {
- return getWidthOrHeight( elem, name, extra );
- } ) :
- getWidthOrHeight( elem, name, extra );
- }
- },
-
- set: function( elem, value, extra ) {
- var matches,
- styles = extra && getStyles( elem ),
- subtract = extra && augmentWidthOrHeight(
- elem,
- name,
- extra,
- jQuery.css( elem, "boxSizing", false, styles ) === "border-box",
- styles
- );
-
- // Convert to pixels if value adjustment is needed
- if ( subtract && ( matches = rcssNum.exec( value ) ) &&
- ( matches[ 3 ] || "px" ) !== "px" ) {
-
- elem.style[ name ] = value;
- value = jQuery.css( elem, name );
- }
-
- return setPositiveNumber( elem, value, subtract );
- }
- };
-} );
-
-jQuery.cssHooks.marginLeft = addGetHookIf( support.reliableMarginLeft,
- function( elem, computed ) {
- if ( computed ) {
- return ( parseFloat( curCSS( elem, "marginLeft" ) ) ||
- elem.getBoundingClientRect().left -
- swap( elem, { marginLeft: 0 }, function() {
- return elem.getBoundingClientRect().left;
- } )
- ) + "px";
- }
- }
-);
-
-// These hooks are used by animate to expand properties
-jQuery.each( {
- margin: "",
- padding: "",
- border: "Width"
-}, function( prefix, suffix ) {
- jQuery.cssHooks[ prefix + suffix ] = {
- expand: function( value ) {
- var i = 0,
- expanded = {},
-
- // Assumes a single number if not a string
- parts = typeof value === "string" ? value.split( " " ) : [ value ];
-
- for ( ; i < 4; i++ ) {
- expanded[ prefix + cssExpand[ i ] + suffix ] =
- parts[ i ] || parts[ i - 2 ] || parts[ 0 ];
- }
-
- return expanded;
- }
- };
-
- if ( !rmargin.test( prefix ) ) {
- jQuery.cssHooks[ prefix + suffix ].set = setPositiveNumber;
- }
-} );
-
-jQuery.fn.extend( {
- css: function( name, value ) {
- return access( this, function( elem, name, value ) {
- var styles, len,
- map = {},
- i = 0;
-
- if ( Array.isArray( name ) ) {
- styles = getStyles( elem );
- len = name.length;
-
- for ( ; i < len; i++ ) {
- map[ name[ i ] ] = jQuery.css( elem, name[ i ], false, styles );
- }
-
- return map;
- }
-
- return value !== undefined ?
- jQuery.style( elem, name, value ) :
- jQuery.css( elem, name );
- }, name, value, arguments.length > 1 );
- }
-} );
-
-
-function Tween( elem, options, prop, end, easing ) {
- return new Tween.prototype.init( elem, options, prop, end, easing );
-}
-jQuery.Tween = Tween;
-
-Tween.prototype = {
- constructor: Tween,
- init: function( elem, options, prop, end, easing, unit ) {
- this.elem = elem;
- this.prop = prop;
- this.easing = easing || jQuery.easing._default;
- this.options = options;
- this.start = this.now = this.cur();
- this.end = end;
- this.unit = unit || ( jQuery.cssNumber[ prop ] ? "" : "px" );
- },
- cur: function() {
- var hooks = Tween.propHooks[ this.prop ];
-
- return hooks && hooks.get ?
- hooks.get( this ) :
- Tween.propHooks._default.get( this );
- },
- run: function( percent ) {
- var eased,
- hooks = Tween.propHooks[ this.prop ];
-
- if ( this.options.duration ) {
- this.pos = eased = jQuery.easing[ this.easing ](
- percent, this.options.duration * percent, 0, 1, this.options.duration
- );
- } else {
- this.pos = eased = percent;
- }
- this.now = ( this.end - this.start ) * eased + this.start;
-
- if ( this.options.step ) {
- this.options.step.call( this.elem, this.now, this );
- }
-
- if ( hooks && hooks.set ) {
- hooks.set( this );
- } else {
- Tween.propHooks._default.set( this );
- }
- return this;
- }
-};
-
-Tween.prototype.init.prototype = Tween.prototype;
-
-Tween.propHooks = {
- _default: {
- get: function( tween ) {
- var result;
-
- // Use a property on the element directly when it is not a DOM element,
- // or when there is no matching style property that exists.
- if ( tween.elem.nodeType !== 1 ||
- tween.elem[ tween.prop ] != null && tween.elem.style[ tween.prop ] == null ) {
- return tween.elem[ tween.prop ];
- }
-
- // Passing an empty string as a 3rd parameter to .css will automatically
- // attempt a parseFloat and fallback to a string if the parse fails.
- // Simple values such as "10px" are parsed to Float;
- // complex values such as "rotate(1rad)" are returned as-is.
- result = jQuery.css( tween.elem, tween.prop, "" );
-
- // Empty strings, null, undefined and "auto" are converted to 0.
- return !result || result === "auto" ? 0 : result;
- },
- set: function( tween ) {
-
- // Use step hook for back compat.
- // Use cssHook if its there.
- // Use .style if available and use plain properties where available.
- if ( jQuery.fx.step[ tween.prop ] ) {
- jQuery.fx.step[ tween.prop ]( tween );
- } else if ( tween.elem.nodeType === 1 &&
- ( tween.elem.style[ jQuery.cssProps[ tween.prop ] ] != null ||
- jQuery.cssHooks[ tween.prop ] ) ) {
- jQuery.style( tween.elem, tween.prop, tween.now + tween.unit );
- } else {
- tween.elem[ tween.prop ] = tween.now;
- }
- }
- }
-};
-
-// Support: IE <=9 only
-// Panic based approach to setting things on disconnected nodes
-Tween.propHooks.scrollTop = Tween.propHooks.scrollLeft = {
- set: function( tween ) {
- if ( tween.elem.nodeType && tween.elem.parentNode ) {
- tween.elem[ tween.prop ] = tween.now;
- }
- }
-};
-
-jQuery.easing = {
- linear: function( p ) {
- return p;
- },
- swing: function( p ) {
- return 0.5 - Math.cos( p * Math.PI ) / 2;
- },
- _default: "swing"
-};
-
-jQuery.fx = Tween.prototype.init;
-
-// Back compat <1.8 extension point
-jQuery.fx.step = {};
-
-
-
-
-var
- fxNow, inProgress,
- rfxtypes = /^(?:toggle|show|hide)$/,
- rrun = /queueHooks$/;
-
-function schedule() {
- if ( inProgress ) {
- if ( document.hidden === false && window.requestAnimationFrame ) {
- window.requestAnimationFrame( schedule );
- } else {
- window.setTimeout( schedule, jQuery.fx.interval );
- }
-
- jQuery.fx.tick();
- }
-}
-
-// Animations created synchronously will run synchronously
-function createFxNow() {
- window.setTimeout( function() {
- fxNow = undefined;
- } );
- return ( fxNow = jQuery.now() );
-}
-
-// Generate parameters to create a standard animation
-function genFx( type, includeWidth ) {
- var which,
- i = 0,
- attrs = { height: type };
-
- // If we include width, step value is 1 to do all cssExpand values,
- // otherwise step value is 2 to skip over Left and Right
- includeWidth = includeWidth ? 1 : 0;
- for ( ; i < 4; i += 2 - includeWidth ) {
- which = cssExpand[ i ];
- attrs[ "margin" + which ] = attrs[ "padding" + which ] = type;
- }
-
- if ( includeWidth ) {
- attrs.opacity = attrs.width = type;
- }
-
- return attrs;
-}
-
-function createTween( value, prop, animation ) {
- var tween,
- collection = ( Animation.tweeners[ prop ] || [] ).concat( Animation.tweeners[ "*" ] ),
- index = 0,
- length = collection.length;
- for ( ; index < length; index++ ) {
- if ( ( tween = collection[ index ].call( animation, prop, value ) ) ) {
-
- // We're done with this property
- return tween;
- }
- }
-}
-
-function defaultPrefilter( elem, props, opts ) {
- var prop, value, toggle, hooks, oldfire, propTween, restoreDisplay, display,
- isBox = "width" in props || "height" in props,
- anim = this,
- orig = {},
- style = elem.style,
- hidden = elem.nodeType && isHiddenWithinTree( elem ),
- dataShow = dataPriv.get( elem, "fxshow" );
-
- // Queue-skipping animations hijack the fx hooks
- if ( !opts.queue ) {
- hooks = jQuery._queueHooks( elem, "fx" );
- if ( hooks.unqueued == null ) {
- hooks.unqueued = 0;
- oldfire = hooks.empty.fire;
- hooks.empty.fire = function() {
- if ( !hooks.unqueued ) {
- oldfire();
- }
- };
- }
- hooks.unqueued++;
-
- anim.always( function() {
-
- // Ensure the complete handler is called before this completes
- anim.always( function() {
- hooks.unqueued--;
- if ( !jQuery.queue( elem, "fx" ).length ) {
- hooks.empty.fire();
- }
- } );
- } );
- }
-
- // Detect show/hide animations
- for ( prop in props ) {
- value = props[ prop ];
- if ( rfxtypes.test( value ) ) {
- delete props[ prop ];
- toggle = toggle || value === "toggle";
- if ( value === ( hidden ? "hide" : "show" ) ) {
-
- // Pretend to be hidden if this is a "show" and
- // there is still data from a stopped show/hide
- if ( value === "show" && dataShow && dataShow[ prop ] !== undefined ) {
- hidden = true;
-
- // Ignore all other no-op show/hide data
- } else {
- continue;
- }
- }
- orig[ prop ] = dataShow && dataShow[ prop ] || jQuery.style( elem, prop );
- }
- }
-
- // Bail out if this is a no-op like .hide().hide()
- propTween = !jQuery.isEmptyObject( props );
- if ( !propTween && jQuery.isEmptyObject( orig ) ) {
- return;
- }
-
- // Restrict "overflow" and "display" styles during box animations
- if ( isBox && elem.nodeType === 1 ) {
-
- // Support: IE <=9 - 11, Edge 12 - 13
- // Record all 3 overflow attributes because IE does not infer the shorthand
- // from identically-valued overflowX and overflowY
- opts.overflow = [ style.overflow, style.overflowX, style.overflowY ];
-
- // Identify a display type, preferring old show/hide data over the CSS cascade
- restoreDisplay = dataShow && dataShow.display;
- if ( restoreDisplay == null ) {
- restoreDisplay = dataPriv.get( elem, "display" );
- }
- display = jQuery.css( elem, "display" );
- if ( display === "none" ) {
- if ( restoreDisplay ) {
- display = restoreDisplay;
- } else {
-
- // Get nonempty value(s) by temporarily forcing visibility
- showHide( [ elem ], true );
- restoreDisplay = elem.style.display || restoreDisplay;
- display = jQuery.css( elem, "display" );
- showHide( [ elem ] );
- }
- }
-
- // Animate inline elements as inline-block
- if ( display === "inline" || display === "inline-block" && restoreDisplay != null ) {
- if ( jQuery.css( elem, "float" ) === "none" ) {
-
- // Restore the original display value at the end of pure show/hide animations
- if ( !propTween ) {
- anim.done( function() {
- style.display = restoreDisplay;
- } );
- if ( restoreDisplay == null ) {
- display = style.display;
- restoreDisplay = display === "none" ? "" : display;
- }
- }
- style.display = "inline-block";
- }
- }
- }
-
- if ( opts.overflow ) {
- style.overflow = "hidden";
- anim.always( function() {
- style.overflow = opts.overflow[ 0 ];
- style.overflowX = opts.overflow[ 1 ];
- style.overflowY = opts.overflow[ 2 ];
- } );
- }
-
- // Implement show/hide animations
- propTween = false;
- for ( prop in orig ) {
-
- // General show/hide setup for this element animation
- if ( !propTween ) {
- if ( dataShow ) {
- if ( "hidden" in dataShow ) {
- hidden = dataShow.hidden;
- }
- } else {
- dataShow = dataPriv.access( elem, "fxshow", { display: restoreDisplay } );
- }
-
- // Store hidden/visible for toggle so `.stop().toggle()` "reverses"
- if ( toggle ) {
- dataShow.hidden = !hidden;
- }
-
- // Show elements before animating them
- if ( hidden ) {
- showHide( [ elem ], true );
- }
-
- /* eslint-disable no-loop-func */
-
- anim.done( function() {
-
- /* eslint-enable no-loop-func */
-
- // The final step of a "hide" animation is actually hiding the element
- if ( !hidden ) {
- showHide( [ elem ] );
- }
- dataPriv.remove( elem, "fxshow" );
- for ( prop in orig ) {
- jQuery.style( elem, prop, orig[ prop ] );
- }
- } );
- }
-
- // Per-property setup
- propTween = createTween( hidden ? dataShow[ prop ] : 0, prop, anim );
- if ( !( prop in dataShow ) ) {
- dataShow[ prop ] = propTween.start;
- if ( hidden ) {
- propTween.end = propTween.start;
- propTween.start = 0;
- }
- }
- }
-}
-
-function propFilter( props, specialEasing ) {
- var index, name, easing, value, hooks;
-
- // camelCase, specialEasing and expand cssHook pass
- for ( index in props ) {
- name = jQuery.camelCase( index );
- easing = specialEasing[ name ];
- value = props[ index ];
- if ( Array.isArray( value ) ) {
- easing = value[ 1 ];
- value = props[ index ] = value[ 0 ];
- }
-
- if ( index !== name ) {
- props[ name ] = value;
- delete props[ index ];
- }
-
- hooks = jQuery.cssHooks[ name ];
- if ( hooks && "expand" in hooks ) {
- value = hooks.expand( value );
- delete props[ name ];
-
- // Not quite $.extend, this won't overwrite existing keys.
- // Reusing 'index' because we have the correct "name"
- for ( index in value ) {
- if ( !( index in props ) ) {
- props[ index ] = value[ index ];
- specialEasing[ index ] = easing;
- }
- }
- } else {
- specialEasing[ name ] = easing;
- }
- }
-}
-
-function Animation( elem, properties, options ) {
- var result,
- stopped,
- index = 0,
- length = Animation.prefilters.length,
- deferred = jQuery.Deferred().always( function() {
-
- // Don't match elem in the :animated selector
- delete tick.elem;
- } ),
- tick = function() {
- if ( stopped ) {
- return false;
- }
- var currentTime = fxNow || createFxNow(),
- remaining = Math.max( 0, animation.startTime + animation.duration - currentTime ),
-
- // Support: Android 2.3 only
- // Archaic crash bug won't allow us to use `1 - ( 0.5 || 0 )` (#12497)
- temp = remaining / animation.duration || 0,
- percent = 1 - temp,
- index = 0,
- length = animation.tweens.length;
-
- for ( ; index < length; index++ ) {
- animation.tweens[ index ].run( percent );
- }
-
- deferred.notifyWith( elem, [ animation, percent, remaining ] );
-
- // If there's more to do, yield
- if ( percent < 1 && length ) {
- return remaining;
- }
-
- // If this was an empty animation, synthesize a final progress notification
- if ( !length ) {
- deferred.notifyWith( elem, [ animation, 1, 0 ] );
- }
-
- // Resolve the animation and report its conclusion
- deferred.resolveWith( elem, [ animation ] );
- return false;
- },
- animation = deferred.promise( {
- elem: elem,
- props: jQuery.extend( {}, properties ),
- opts: jQuery.extend( true, {
- specialEasing: {},
- easing: jQuery.easing._default
- }, options ),
- originalProperties: properties,
- originalOptions: options,
- startTime: fxNow || createFxNow(),
- duration: options.duration,
- tweens: [],
- createTween: function( prop, end ) {
- var tween = jQuery.Tween( elem, animation.opts, prop, end,
- animation.opts.specialEasing[ prop ] || animation.opts.easing );
- animation.tweens.push( tween );
- return tween;
- },
- stop: function( gotoEnd ) {
- var index = 0,
-
- // If we are going to the end, we want to run all the tweens
- // otherwise we skip this part
- length = gotoEnd ? animation.tweens.length : 0;
- if ( stopped ) {
- return this;
- }
- stopped = true;
- for ( ; index < length; index++ ) {
- animation.tweens[ index ].run( 1 );
- }
-
- // Resolve when we played the last frame; otherwise, reject
- if ( gotoEnd ) {
- deferred.notifyWith( elem, [ animation, 1, 0 ] );
- deferred.resolveWith( elem, [ animation, gotoEnd ] );
- } else {
- deferred.rejectWith( elem, [ animation, gotoEnd ] );
- }
- return this;
- }
- } ),
- props = animation.props;
-
- propFilter( props, animation.opts.specialEasing );
-
- for ( ; index < length; index++ ) {
- result = Animation.prefilters[ index ].call( animation, elem, props, animation.opts );
- if ( result ) {
- if ( jQuery.isFunction( result.stop ) ) {
- jQuery._queueHooks( animation.elem, animation.opts.queue ).stop =
- jQuery.proxy( result.stop, result );
- }
- return result;
- }
- }
-
- jQuery.map( props, createTween, animation );
-
- if ( jQuery.isFunction( animation.opts.start ) ) {
- animation.opts.start.call( elem, animation );
- }
-
- // Attach callbacks from options
- animation
- .progress( animation.opts.progress )
- .done( animation.opts.done, animation.opts.complete )
- .fail( animation.opts.fail )
- .always( animation.opts.always );
-
- jQuery.fx.timer(
- jQuery.extend( tick, {
- elem: elem,
- anim: animation,
- queue: animation.opts.queue
- } )
- );
-
- return animation;
-}
-
-jQuery.Animation = jQuery.extend( Animation, {
-
- tweeners: {
- "*": [ function( prop, value ) {
- var tween = this.createTween( prop, value );
- adjustCSS( tween.elem, prop, rcssNum.exec( value ), tween );
- return tween;
- } ]
- },
-
- tweener: function( props, callback ) {
- if ( jQuery.isFunction( props ) ) {
- callback = props;
- props = [ "*" ];
- } else {
- props = props.match( rnothtmlwhite );
- }
-
- var prop,
- index = 0,
- length = props.length;
-
- for ( ; index < length; index++ ) {
- prop = props[ index ];
- Animation.tweeners[ prop ] = Animation.tweeners[ prop ] || [];
- Animation.tweeners[ prop ].unshift( callback );
- }
- },
-
- prefilters: [ defaultPrefilter ],
-
- prefilter: function( callback, prepend ) {
- if ( prepend ) {
- Animation.prefilters.unshift( callback );
- } else {
- Animation.prefilters.push( callback );
- }
- }
-} );
-
-jQuery.speed = function( speed, easing, fn ) {
- var opt = speed && typeof speed === "object" ? jQuery.extend( {}, speed ) : {
- complete: fn || !fn && easing ||
- jQuery.isFunction( speed ) && speed,
- duration: speed,
- easing: fn && easing || easing && !jQuery.isFunction( easing ) && easing
- };
-
- // Go to the end state if fx are off
- if ( jQuery.fx.off ) {
- opt.duration = 0;
-
- } else {
- if ( typeof opt.duration !== "number" ) {
- if ( opt.duration in jQuery.fx.speeds ) {
- opt.duration = jQuery.fx.speeds[ opt.duration ];
-
- } else {
- opt.duration = jQuery.fx.speeds._default;
- }
- }
- }
-
- // Normalize opt.queue - true/undefined/null -> "fx"
- if ( opt.queue == null || opt.queue === true ) {
- opt.queue = "fx";
- }
-
- // Queueing
- opt.old = opt.complete;
-
- opt.complete = function() {
- if ( jQuery.isFunction( opt.old ) ) {
- opt.old.call( this );
- }
-
- if ( opt.queue ) {
- jQuery.dequeue( this, opt.queue );
- }
- };
-
- return opt;
-};
-
-jQuery.fn.extend( {
- fadeTo: function( speed, to, easing, callback ) {
-
- // Show any hidden elements after setting opacity to 0
- return this.filter( isHiddenWithinTree ).css( "opacity", 0 ).show()
-
- // Animate to the value specified
- .end().animate( { opacity: to }, speed, easing, callback );
- },
- animate: function( prop, speed, easing, callback ) {
- var empty = jQuery.isEmptyObject( prop ),
- optall = jQuery.speed( speed, easing, callback ),
- doAnimation = function() {
-
- // Operate on a copy of prop so per-property easing won't be lost
- var anim = Animation( this, jQuery.extend( {}, prop ), optall );
-
- // Empty animations, or finishing resolves immediately
- if ( empty || dataPriv.get( this, "finish" ) ) {
- anim.stop( true );
- }
- };
- doAnimation.finish = doAnimation;
-
- return empty || optall.queue === false ?
- this.each( doAnimation ) :
- this.queue( optall.queue, doAnimation );
- },
- stop: function( type, clearQueue, gotoEnd ) {
- var stopQueue = function( hooks ) {
- var stop = hooks.stop;
- delete hooks.stop;
- stop( gotoEnd );
- };
-
- if ( typeof type !== "string" ) {
- gotoEnd = clearQueue;
- clearQueue = type;
- type = undefined;
- }
- if ( clearQueue && type !== false ) {
- this.queue( type || "fx", [] );
- }
-
- return this.each( function() {
- var dequeue = true,
- index = type != null && type + "queueHooks",
- timers = jQuery.timers,
- data = dataPriv.get( this );
-
- if ( index ) {
- if ( data[ index ] && data[ index ].stop ) {
- stopQueue( data[ index ] );
- }
- } else {
- for ( index in data ) {
- if ( data[ index ] && data[ index ].stop && rrun.test( index ) ) {
- stopQueue( data[ index ] );
- }
- }
- }
-
- for ( index = timers.length; index--; ) {
- if ( timers[ index ].elem === this &&
- ( type == null || timers[ index ].queue === type ) ) {
-
- timers[ index ].anim.stop( gotoEnd );
- dequeue = false;
- timers.splice( index, 1 );
- }
- }
-
- // Start the next in the queue if the last step wasn't forced.
- // Timers currently will call their complete callbacks, which
- // will dequeue but only if they were gotoEnd.
- if ( dequeue || !gotoEnd ) {
- jQuery.dequeue( this, type );
- }
- } );
- },
- finish: function( type ) {
- if ( type !== false ) {
- type = type || "fx";
- }
- return this.each( function() {
- var index,
- data = dataPriv.get( this ),
- queue = data[ type + "queue" ],
- hooks = data[ type + "queueHooks" ],
- timers = jQuery.timers,
- length = queue ? queue.length : 0;
-
- // Enable finishing flag on private data
- data.finish = true;
-
- // Empty the queue first
- jQuery.queue( this, type, [] );
-
- if ( hooks && hooks.stop ) {
- hooks.stop.call( this, true );
- }
-
- // Look for any active animations, and finish them
- for ( index = timers.length; index--; ) {
- if ( timers[ index ].elem === this && timers[ index ].queue === type ) {
- timers[ index ].anim.stop( true );
- timers.splice( index, 1 );
- }
- }
-
- // Look for any animations in the old queue and finish them
- for ( index = 0; index < length; index++ ) {
- if ( queue[ index ] && queue[ index ].finish ) {
- queue[ index ].finish.call( this );
- }
- }
-
- // Turn off finishing flag
- delete data.finish;
- } );
- }
-} );
-
-jQuery.each( [ "toggle", "show", "hide" ], function( i, name ) {
- var cssFn = jQuery.fn[ name ];
- jQuery.fn[ name ] = function( speed, easing, callback ) {
- return speed == null || typeof speed === "boolean" ?
- cssFn.apply( this, arguments ) :
- this.animate( genFx( name, true ), speed, easing, callback );
- };
-} );
-
-// Generate shortcuts for custom animations
-jQuery.each( {
- slideDown: genFx( "show" ),
- slideUp: genFx( "hide" ),
- slideToggle: genFx( "toggle" ),
- fadeIn: { opacity: "show" },
- fadeOut: { opacity: "hide" },
- fadeToggle: { opacity: "toggle" }
-}, function( name, props ) {
- jQuery.fn[ name ] = function( speed, easing, callback ) {
- return this.animate( props, speed, easing, callback );
- };
-} );
-
-jQuery.timers = [];
-jQuery.fx.tick = function() {
- var timer,
- i = 0,
- timers = jQuery.timers;
-
- fxNow = jQuery.now();
-
- for ( ; i < timers.length; i++ ) {
- timer = timers[ i ];
-
- // Run the timer and safely remove it when done (allowing for external removal)
- if ( !timer() && timers[ i ] === timer ) {
- timers.splice( i--, 1 );
- }
- }
-
- if ( !timers.length ) {
- jQuery.fx.stop();
- }
- fxNow = undefined;
-};
-
-jQuery.fx.timer = function( timer ) {
- jQuery.timers.push( timer );
- jQuery.fx.start();
-};
-
-jQuery.fx.interval = 13;
-jQuery.fx.start = function() {
- if ( inProgress ) {
- return;
- }
-
- inProgress = true;
- schedule();
-};
-
-jQuery.fx.stop = function() {
- inProgress = null;
-};
-
-jQuery.fx.speeds = {
- slow: 600,
- fast: 200,
-
- // Default speed
- _default: 400
-};
-
-
-// Based off of the plugin by Clint Helfers, with permission.
-// https://web.archive.org/web/20100324014747/https://blindsignals.com/index.php/2009/07/jquery-delay/
-jQuery.fn.delay = function( time, type ) {
- time = jQuery.fx ? jQuery.fx.speeds[ time ] || time : time;
- type = type || "fx";
-
- return this.queue( type, function( next, hooks ) {
- var timeout = window.setTimeout( next, time );
- hooks.stop = function() {
- window.clearTimeout( timeout );
- };
- } );
-};
-
-
-( function() {
- var input = document.createElement( "input" ),
- select = document.createElement( "select" ),
- opt = select.appendChild( document.createElement( "option" ) );
-
- input.type = "checkbox";
-
- // Support: Android <=4.3 only
- // Default value for a checkbox should be "on"
- support.checkOn = input.value !== "";
-
- // Support: IE <=11 only
- // Must access selectedIndex to make default options select
- support.optSelected = opt.selected;
-
- // Support: IE <=11 only
- // An input loses its value after becoming a radio
- input = document.createElement( "input" );
- input.value = "t";
- input.type = "radio";
- support.radioValue = input.value === "t";
-} )();
-
-
-var boolHook,
- attrHandle = jQuery.expr.attrHandle;
-
-jQuery.fn.extend( {
- attr: function( name, value ) {
- return access( this, jQuery.attr, name, value, arguments.length > 1 );
- },
-
- removeAttr: function( name ) {
- return this.each( function() {
- jQuery.removeAttr( this, name );
- } );
- }
-} );
-
-jQuery.extend( {
- attr: function( elem, name, value ) {
- var ret, hooks,
- nType = elem.nodeType;
-
- // Don't get/set attributes on text, comment and attribute nodes
- if ( nType === 3 || nType === 8 || nType === 2 ) {
- return;
- }
-
- // Fallback to prop when attributes are not supported
- if ( typeof elem.getAttribute === "undefined" ) {
- return jQuery.prop( elem, name, value );
- }
-
- // Attribute hooks are determined by the lowercase version
- // Grab necessary hook if one is defined
- if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) {
- hooks = jQuery.attrHooks[ name.toLowerCase() ] ||
- ( jQuery.expr.match.bool.test( name ) ? boolHook : undefined );
- }
-
- if ( value !== undefined ) {
- if ( value === null ) {
- jQuery.removeAttr( elem, name );
- return;
- }
-
- if ( hooks && "set" in hooks &&
- ( ret = hooks.set( elem, value, name ) ) !== undefined ) {
- return ret;
- }
-
- elem.setAttribute( name, value + "" );
- return value;
- }
-
- if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) {
- return ret;
- }
-
- ret = jQuery.find.attr( elem, name );
-
- // Non-existent attributes return null, we normalize to undefined
- return ret == null ? undefined : ret;
- },
-
- attrHooks: {
- type: {
- set: function( elem, value ) {
- if ( !support.radioValue && value === "radio" &&
- nodeName( elem, "input" ) ) {
- var val = elem.value;
- elem.setAttribute( "type", value );
- if ( val ) {
- elem.value = val;
- }
- return value;
- }
- }
- }
- },
-
- removeAttr: function( elem, value ) {
- var name,
- i = 0,
-
- // Attribute names can contain non-HTML whitespace characters
- // https://html.spec.whatwg.org/multipage/syntax.html#attributes-2
- attrNames = value && value.match( rnothtmlwhite );
-
- if ( attrNames && elem.nodeType === 1 ) {
- while ( ( name = attrNames[ i++ ] ) ) {
- elem.removeAttribute( name );
- }
- }
- }
-} );
-
-// Hooks for boolean attributes
-boolHook = {
- set: function( elem, value, name ) {
- if ( value === false ) {
-
- // Remove boolean attributes when set to false
- jQuery.removeAttr( elem, name );
- } else {
- elem.setAttribute( name, name );
- }
- return name;
- }
-};
-
-jQuery.each( jQuery.expr.match.bool.source.match( /\w+/g ), function( i, name ) {
- var getter = attrHandle[ name ] || jQuery.find.attr;
-
- attrHandle[ name ] = function( elem, name, isXML ) {
- var ret, handle,
- lowercaseName = name.toLowerCase();
-
- if ( !isXML ) {
-
- // Avoid an infinite loop by temporarily removing this function from the getter
- handle = attrHandle[ lowercaseName ];
- attrHandle[ lowercaseName ] = ret;
- ret = getter( elem, name, isXML ) != null ?
- lowercaseName :
- null;
- attrHandle[ lowercaseName ] = handle;
- }
- return ret;
- };
-} );
-
-
-
-
-var rfocusable = /^(?:input|select|textarea|button)$/i,
- rclickable = /^(?:a|area)$/i;
-
-jQuery.fn.extend( {
- prop: function( name, value ) {
- return access( this, jQuery.prop, name, value, arguments.length > 1 );
- },
-
- removeProp: function( name ) {
- return this.each( function() {
- delete this[ jQuery.propFix[ name ] || name ];
- } );
- }
-} );
-
-jQuery.extend( {
- prop: function( elem, name, value ) {
- var ret, hooks,
- nType = elem.nodeType;
-
- // Don't get/set properties on text, comment and attribute nodes
- if ( nType === 3 || nType === 8 || nType === 2 ) {
- return;
- }
-
- if ( nType !== 1 || !jQuery.isXMLDoc( elem ) ) {
-
- // Fix name and attach hooks
- name = jQuery.propFix[ name ] || name;
- hooks = jQuery.propHooks[ name ];
- }
-
- if ( value !== undefined ) {
- if ( hooks && "set" in hooks &&
- ( ret = hooks.set( elem, value, name ) ) !== undefined ) {
- return ret;
- }
-
- return ( elem[ name ] = value );
- }
-
- if ( hooks && "get" in hooks && ( ret = hooks.get( elem, name ) ) !== null ) {
- return ret;
- }
-
- return elem[ name ];
- },
-
- propHooks: {
- tabIndex: {
- get: function( elem ) {
-
- // Support: IE <=9 - 11 only
- // elem.tabIndex doesn't always return the
- // correct value when it hasn't been explicitly set
- // https://web.archive.org/web/20141116233347/https://fluidproject.org/blog/2008/01/09/getting-setting-and-removing-tabindex-values-with-javascript/
- // Use proper attribute retrieval(#12072)
- var tabindex = jQuery.find.attr( elem, "tabindex" );
-
- if ( tabindex ) {
- return parseInt( tabindex, 10 );
- }
-
- if (
- rfocusable.test( elem.nodeName ) ||
- rclickable.test( elem.nodeName ) &&
- elem.href
- ) {
- return 0;
- }
-
- return -1;
- }
- }
- },
-
- propFix: {
- "for": "htmlFor",
- "class": "className"
- }
-} );
-
-// Support: IE <=11 only
-// Accessing the selectedIndex property
-// forces the browser to respect setting selected
-// on the option
-// The getter ensures a default option is selected
-// when in an optgroup
-// eslint rule "no-unused-expressions" is disabled for this code
-// since it considers such accessions noop
-if ( !support.optSelected ) {
- jQuery.propHooks.selected = {
- get: function( elem ) {
-
- /* eslint no-unused-expressions: "off" */
-
- var parent = elem.parentNode;
- if ( parent && parent.parentNode ) {
- parent.parentNode.selectedIndex;
- }
- return null;
- },
- set: function( elem ) {
-
- /* eslint no-unused-expressions: "off" */
-
- var parent = elem.parentNode;
- if ( parent ) {
- parent.selectedIndex;
-
- if ( parent.parentNode ) {
- parent.parentNode.selectedIndex;
- }
- }
- }
- };
-}
-
-jQuery.each( [
- "tabIndex",
- "readOnly",
- "maxLength",
- "cellSpacing",
- "cellPadding",
- "rowSpan",
- "colSpan",
- "useMap",
- "frameBorder",
- "contentEditable"
-], function() {
- jQuery.propFix[ this.toLowerCase() ] = this;
-} );
-
-
-
-
- // Strip and collapse whitespace according to HTML spec
- // https://html.spec.whatwg.org/multipage/infrastructure.html#strip-and-collapse-whitespace
- function stripAndCollapse( value ) {
- var tokens = value.match( rnothtmlwhite ) || [];
- return tokens.join( " " );
- }
-
-
-function getClass( elem ) {
- return elem.getAttribute && elem.getAttribute( "class" ) || "";
-}
-
-jQuery.fn.extend( {
- addClass: function( value ) {
- var classes, elem, cur, curValue, clazz, j, finalValue,
- i = 0;
-
- if ( jQuery.isFunction( value ) ) {
- return this.each( function( j ) {
- jQuery( this ).addClass( value.call( this, j, getClass( this ) ) );
- } );
- }
-
- if ( typeof value === "string" && value ) {
- classes = value.match( rnothtmlwhite ) || [];
-
- while ( ( elem = this[ i++ ] ) ) {
- curValue = getClass( elem );
- cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " );
-
- if ( cur ) {
- j = 0;
- while ( ( clazz = classes[ j++ ] ) ) {
- if ( cur.indexOf( " " + clazz + " " ) < 0 ) {
- cur += clazz + " ";
- }
- }
-
- // Only assign if different to avoid unneeded rendering.
- finalValue = stripAndCollapse( cur );
- if ( curValue !== finalValue ) {
- elem.setAttribute( "class", finalValue );
- }
- }
- }
- }
-
- return this;
- },
-
- removeClass: function( value ) {
- var classes, elem, cur, curValue, clazz, j, finalValue,
- i = 0;
-
- if ( jQuery.isFunction( value ) ) {
- return this.each( function( j ) {
- jQuery( this ).removeClass( value.call( this, j, getClass( this ) ) );
- } );
- }
-
- if ( !arguments.length ) {
- return this.attr( "class", "" );
- }
-
- if ( typeof value === "string" && value ) {
- classes = value.match( rnothtmlwhite ) || [];
-
- while ( ( elem = this[ i++ ] ) ) {
- curValue = getClass( elem );
-
- // This expression is here for better compressibility (see addClass)
- cur = elem.nodeType === 1 && ( " " + stripAndCollapse( curValue ) + " " );
-
- if ( cur ) {
- j = 0;
- while ( ( clazz = classes[ j++ ] ) ) {
-
- // Remove *all* instances
- while ( cur.indexOf( " " + clazz + " " ) > -1 ) {
- cur = cur.replace( " " + clazz + " ", " " );
- }
- }
-
- // Only assign if different to avoid unneeded rendering.
- finalValue = stripAndCollapse( cur );
- if ( curValue !== finalValue ) {
- elem.setAttribute( "class", finalValue );
- }
- }
- }
- }
-
- return this;
- },
-
- toggleClass: function( value, stateVal ) {
- var type = typeof value;
-
- if ( typeof stateVal === "boolean" && type === "string" ) {
- return stateVal ? this.addClass( value ) : this.removeClass( value );
- }
-
- if ( jQuery.isFunction( value ) ) {
- return this.each( function( i ) {
- jQuery( this ).toggleClass(
- value.call( this, i, getClass( this ), stateVal ),
- stateVal
- );
- } );
- }
-
- return this.each( function() {
- var className, i, self, classNames;
-
- if ( type === "string" ) {
-
- // Toggle individual class names
- i = 0;
- self = jQuery( this );
- classNames = value.match( rnothtmlwhite ) || [];
-
- while ( ( className = classNames[ i++ ] ) ) {
-
- // Check each className given, space separated list
- if ( self.hasClass( className ) ) {
- self.removeClass( className );
- } else {
- self.addClass( className );
- }
- }
-
- // Toggle whole class name
- } else if ( value === undefined || type === "boolean" ) {
- className = getClass( this );
- if ( className ) {
-
- // Store className if set
- dataPriv.set( this, "__className__", className );
- }
-
- // If the element has a class name or if we're passed `false`,
- // then remove the whole classname (if there was one, the above saved it).
- // Otherwise bring back whatever was previously saved (if anything),
- // falling back to the empty string if nothing was stored.
- if ( this.setAttribute ) {
- this.setAttribute( "class",
- className || value === false ?
- "" :
- dataPriv.get( this, "__className__" ) || ""
- );
- }
- }
- } );
- },
-
- hasClass: function( selector ) {
- var className, elem,
- i = 0;
-
- className = " " + selector + " ";
- while ( ( elem = this[ i++ ] ) ) {
- if ( elem.nodeType === 1 &&
- ( " " + stripAndCollapse( getClass( elem ) ) + " " ).indexOf( className ) > -1 ) {
- return true;
- }
- }
-
- return false;
- }
-} );
-
-
-
-
-var rreturn = /\r/g;
-
-jQuery.fn.extend( {
- val: function( value ) {
- var hooks, ret, isFunction,
- elem = this[ 0 ];
-
- if ( !arguments.length ) {
- if ( elem ) {
- hooks = jQuery.valHooks[ elem.type ] ||
- jQuery.valHooks[ elem.nodeName.toLowerCase() ];
-
- if ( hooks &&
- "get" in hooks &&
- ( ret = hooks.get( elem, "value" ) ) !== undefined
- ) {
- return ret;
- }
-
- ret = elem.value;
-
- // Handle most common string cases
- if ( typeof ret === "string" ) {
- return ret.replace( rreturn, "" );
- }
-
- // Handle cases where value is null/undef or number
- return ret == null ? "" : ret;
- }
-
- return;
- }
-
- isFunction = jQuery.isFunction( value );
-
- return this.each( function( i ) {
- var val;
-
- if ( this.nodeType !== 1 ) {
- return;
- }
-
- if ( isFunction ) {
- val = value.call( this, i, jQuery( this ).val() );
- } else {
- val = value;
- }
-
- // Treat null/undefined as ""; convert numbers to string
- if ( val == null ) {
- val = "";
-
- } else if ( typeof val === "number" ) {
- val += "";
-
- } else if ( Array.isArray( val ) ) {
- val = jQuery.map( val, function( value ) {
- return value == null ? "" : value + "";
- } );
- }
-
- hooks = jQuery.valHooks[ this.type ] || jQuery.valHooks[ this.nodeName.toLowerCase() ];
-
- // If set returns undefined, fall back to normal setting
- if ( !hooks || !( "set" in hooks ) || hooks.set( this, val, "value" ) === undefined ) {
- this.value = val;
- }
- } );
- }
-} );
-
-jQuery.extend( {
- valHooks: {
- option: {
- get: function( elem ) {
-
- var val = jQuery.find.attr( elem, "value" );
- return val != null ?
- val :
-
- // Support: IE <=10 - 11 only
- // option.text throws exceptions (#14686, #14858)
- // Strip and collapse whitespace
- // https://html.spec.whatwg.org/#strip-and-collapse-whitespace
- stripAndCollapse( jQuery.text( elem ) );
- }
- },
- select: {
- get: function( elem ) {
- var value, option, i,
- options = elem.options,
- index = elem.selectedIndex,
- one = elem.type === "select-one",
- values = one ? null : [],
- max = one ? index + 1 : options.length;
-
- if ( index < 0 ) {
- i = max;
-
- } else {
- i = one ? index : 0;
- }
-
- // Loop through all the selected options
- for ( ; i < max; i++ ) {
- option = options[ i ];
-
- // Support: IE <=9 only
- // IE8-9 doesn't update selected after form reset (#2551)
- if ( ( option.selected || i === index ) &&
-
- // Don't return options that are disabled or in a disabled optgroup
- !option.disabled &&
- ( !option.parentNode.disabled ||
- !nodeName( option.parentNode, "optgroup" ) ) ) {
-
- // Get the specific value for the option
- value = jQuery( option ).val();
-
- // We don't need an array for one selects
- if ( one ) {
- return value;
- }
-
- // Multi-Selects return an array
- values.push( value );
- }
- }
-
- return values;
- },
-
- set: function( elem, value ) {
- var optionSet, option,
- options = elem.options,
- values = jQuery.makeArray( value ),
- i = options.length;
-
- while ( i-- ) {
- option = options[ i ];
-
- /* eslint-disable no-cond-assign */
-
- if ( option.selected =
- jQuery.inArray( jQuery.valHooks.option.get( option ), values ) > -1
- ) {
- optionSet = true;
- }
-
- /* eslint-enable no-cond-assign */
- }
-
- // Force browsers to behave consistently when non-matching value is set
- if ( !optionSet ) {
- elem.selectedIndex = -1;
- }
- return values;
- }
- }
- }
-} );
-
-// Radios and checkboxes getter/setter
-jQuery.each( [ "radio", "checkbox" ], function() {
- jQuery.valHooks[ this ] = {
- set: function( elem, value ) {
- if ( Array.isArray( value ) ) {
- return ( elem.checked = jQuery.inArray( jQuery( elem ).val(), value ) > -1 );
- }
- }
- };
- if ( !support.checkOn ) {
- jQuery.valHooks[ this ].get = function( elem ) {
- return elem.getAttribute( "value" ) === null ? "on" : elem.value;
- };
- }
-} );
-
-
-
-
-// Return jQuery for attributes-only inclusion
-
-
-var rfocusMorph = /^(?:focusinfocus|focusoutblur)$/;
-
-jQuery.extend( jQuery.event, {
-
- trigger: function( event, data, elem, onlyHandlers ) {
-
- var i, cur, tmp, bubbleType, ontype, handle, special,
- eventPath = [ elem || document ],
- type = hasOwn.call( event, "type" ) ? event.type : event,
- namespaces = hasOwn.call( event, "namespace" ) ? event.namespace.split( "." ) : [];
-
- cur = tmp = elem = elem || document;
-
- // Don't do events on text and comment nodes
- if ( elem.nodeType === 3 || elem.nodeType === 8 ) {
- return;
- }
-
- // focus/blur morphs to focusin/out; ensure we're not firing them right now
- if ( rfocusMorph.test( type + jQuery.event.triggered ) ) {
- return;
- }
-
- if ( type.indexOf( "." ) > -1 ) {
-
- // Namespaced trigger; create a regexp to match event type in handle()
- namespaces = type.split( "." );
- type = namespaces.shift();
- namespaces.sort();
- }
- ontype = type.indexOf( ":" ) < 0 && "on" + type;
-
- // Caller can pass in a jQuery.Event object, Object, or just an event type string
- event = event[ jQuery.expando ] ?
- event :
- new jQuery.Event( type, typeof event === "object" && event );
-
- // Trigger bitmask: & 1 for native handlers; & 2 for jQuery (always true)
- event.isTrigger = onlyHandlers ? 2 : 3;
- event.namespace = namespaces.join( "." );
- event.rnamespace = event.namespace ?
- new RegExp( "(^|\\.)" + namespaces.join( "\\.(?:.*\\.|)" ) + "(\\.|$)" ) :
- null;
-
- // Clean up the event in case it is being reused
- event.result = undefined;
- if ( !event.target ) {
- event.target = elem;
- }
-
- // Clone any incoming data and prepend the event, creating the handler arg list
- data = data == null ?
- [ event ] :
- jQuery.makeArray( data, [ event ] );
-
- // Allow special events to draw outside the lines
- special = jQuery.event.special[ type ] || {};
- if ( !onlyHandlers && special.trigger && special.trigger.apply( elem, data ) === false ) {
- return;
- }
-
- // Determine event propagation path in advance, per W3C events spec (#9951)
- // Bubble up to document, then to window; watch for a global ownerDocument var (#9724)
- if ( !onlyHandlers && !special.noBubble && !jQuery.isWindow( elem ) ) {
-
- bubbleType = special.delegateType || type;
- if ( !rfocusMorph.test( bubbleType + type ) ) {
- cur = cur.parentNode;
- }
- for ( ; cur; cur = cur.parentNode ) {
- eventPath.push( cur );
- tmp = cur;
- }
-
- // Only add window if we got to document (e.g., not plain obj or detached DOM)
- if ( tmp === ( elem.ownerDocument || document ) ) {
- eventPath.push( tmp.defaultView || tmp.parentWindow || window );
- }
- }
-
- // Fire handlers on the event path
- i = 0;
- while ( ( cur = eventPath[ i++ ] ) && !event.isPropagationStopped() ) {
-
- event.type = i > 1 ?
- bubbleType :
- special.bindType || type;
-
- // jQuery handler
- handle = ( dataPriv.get( cur, "events" ) || {} )[ event.type ] &&
- dataPriv.get( cur, "handle" );
- if ( handle ) {
- handle.apply( cur, data );
- }
-
- // Native handler
- handle = ontype && cur[ ontype ];
- if ( handle && handle.apply && acceptData( cur ) ) {
- event.result = handle.apply( cur, data );
- if ( event.result === false ) {
- event.preventDefault();
- }
- }
- }
- event.type = type;
-
- // If nobody prevented the default action, do it now
- if ( !onlyHandlers && !event.isDefaultPrevented() ) {
-
- if ( ( !special._default ||
- special._default.apply( eventPath.pop(), data ) === false ) &&
- acceptData( elem ) ) {
-
- // Call a native DOM method on the target with the same name as the event.
- // Don't do default actions on window, that's where global variables be (#6170)
- if ( ontype && jQuery.isFunction( elem[ type ] ) && !jQuery.isWindow( elem ) ) {
-
- // Don't re-trigger an onFOO event when we call its FOO() method
- tmp = elem[ ontype ];
-
- if ( tmp ) {
- elem[ ontype ] = null;
- }
-
- // Prevent re-triggering of the same event, since we already bubbled it above
- jQuery.event.triggered = type;
- elem[ type ]();
- jQuery.event.triggered = undefined;
-
- if ( tmp ) {
- elem[ ontype ] = tmp;
- }
- }
- }
- }
-
- return event.result;
- },
-
- // Piggyback on a donor event to simulate a different one
- // Used only for `focus(in | out)` events
- simulate: function( type, elem, event ) {
- var e = jQuery.extend(
- new jQuery.Event(),
- event,
- {
- type: type,
- isSimulated: true
- }
- );
-
- jQuery.event.trigger( e, null, elem );
- }
-
-} );
-
-jQuery.fn.extend( {
-
- trigger: function( type, data ) {
- return this.each( function() {
- jQuery.event.trigger( type, data, this );
- } );
- },
- triggerHandler: function( type, data ) {
- var elem = this[ 0 ];
- if ( elem ) {
- return jQuery.event.trigger( type, data, elem, true );
- }
- }
-} );
-
-
-jQuery.each( ( "blur focus focusin focusout resize scroll click dblclick " +
- "mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave " +
- "change select submit keydown keypress keyup contextmenu" ).split( " " ),
- function( i, name ) {
-
- // Handle event binding
- jQuery.fn[ name ] = function( data, fn ) {
- return arguments.length > 0 ?
- this.on( name, null, data, fn ) :
- this.trigger( name );
- };
-} );
-
-jQuery.fn.extend( {
- hover: function( fnOver, fnOut ) {
- return this.mouseenter( fnOver ).mouseleave( fnOut || fnOver );
- }
-} );
-
-
-
-
-support.focusin = "onfocusin" in window;
-
-
-// Support: Firefox <=44
-// Firefox doesn't have focus(in | out) events
-// Related ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=687787
-//
-// Support: Chrome <=48 - 49, Safari <=9.0 - 9.1
-// focus(in | out) events fire after focus & blur events,
-// which is spec violation - https://www.w3.org/TR/DOM-Level-3-Events/#events-focusevent-event-order
-// Related ticket - https://bugs.chromium.org/p/chromium/issues/detail?id=449857
-if ( !support.focusin ) {
- jQuery.each( { focus: "focusin", blur: "focusout" }, function( orig, fix ) {
-
- // Attach a single capturing handler on the document while someone wants focusin/focusout
- var handler = function( event ) {
- jQuery.event.simulate( fix, event.target, jQuery.event.fix( event ) );
- };
-
- jQuery.event.special[ fix ] = {
- setup: function() {
- var doc = this.ownerDocument || this,
- attaches = dataPriv.access( doc, fix );
-
- if ( !attaches ) {
- doc.addEventListener( orig, handler, true );
- }
- dataPriv.access( doc, fix, ( attaches || 0 ) + 1 );
- },
- teardown: function() {
- var doc = this.ownerDocument || this,
- attaches = dataPriv.access( doc, fix ) - 1;
-
- if ( !attaches ) {
- doc.removeEventListener( orig, handler, true );
- dataPriv.remove( doc, fix );
-
- } else {
- dataPriv.access( doc, fix, attaches );
- }
- }
- };
- } );
-}
-var location = window.location;
-
-var nonce = jQuery.now();
-
-var rquery = ( /\?/ );
-
-
-
-// Cross-browser xml parsing
-jQuery.parseXML = function( data ) {
- var xml;
- if ( !data || typeof data !== "string" ) {
- return null;
- }
-
- // Support: IE 9 - 11 only
- // IE throws on parseFromString with invalid input.
- try {
- xml = ( new window.DOMParser() ).parseFromString( data, "text/xml" );
- } catch ( e ) {
- xml = undefined;
- }
-
- if ( !xml || xml.getElementsByTagName( "parsererror" ).length ) {
- jQuery.error( "Invalid XML: " + data );
- }
- return xml;
-};
-
-
-var
- rbracket = /\[\]$/,
- rCRLF = /\r?\n/g,
- rsubmitterTypes = /^(?:submit|button|image|reset|file)$/i,
- rsubmittable = /^(?:input|select|textarea|keygen)/i;
-
-function buildParams( prefix, obj, traditional, add ) {
- var name;
-
- if ( Array.isArray( obj ) ) {
-
- // Serialize array item.
- jQuery.each( obj, function( i, v ) {
- if ( traditional || rbracket.test( prefix ) ) {
-
- // Treat each array item as a scalar.
- add( prefix, v );
-
- } else {
-
- // Item is non-scalar (array or object), encode its numeric index.
- buildParams(
- prefix + "[" + ( typeof v === "object" && v != null ? i : "" ) + "]",
- v,
- traditional,
- add
- );
- }
- } );
-
- } else if ( !traditional && jQuery.type( obj ) === "object" ) {
-
- // Serialize object item.
- for ( name in obj ) {
- buildParams( prefix + "[" + name + "]", obj[ name ], traditional, add );
- }
-
- } else {
-
- // Serialize scalar item.
- add( prefix, obj );
- }
-}
-
-// Serialize an array of form elements or a set of
-// key/values into a query string
-jQuery.param = function( a, traditional ) {
- var prefix,
- s = [],
- add = function( key, valueOrFunction ) {
-
- // If value is a function, invoke it and use its return value
- var value = jQuery.isFunction( valueOrFunction ) ?
- valueOrFunction() :
- valueOrFunction;
-
- s[ s.length ] = encodeURIComponent( key ) + "=" +
- encodeURIComponent( value == null ? "" : value );
- };
-
- // If an array was passed in, assume that it is an array of form elements.
- if ( Array.isArray( a ) || ( a.jquery && !jQuery.isPlainObject( a ) ) ) {
-
- // Serialize the form elements
- jQuery.each( a, function() {
- add( this.name, this.value );
- } );
-
- } else {
-
- // If traditional, encode the "old" way (the way 1.3.2 or older
- // did it), otherwise encode params recursively.
- for ( prefix in a ) {
- buildParams( prefix, a[ prefix ], traditional, add );
- }
- }
-
- // Return the resulting serialization
- return s.join( "&" );
-};
-
-jQuery.fn.extend( {
- serialize: function() {
- return jQuery.param( this.serializeArray() );
- },
- serializeArray: function() {
- return this.map( function() {
-
- // Can add propHook for "elements" to filter or add form elements
- var elements = jQuery.prop( this, "elements" );
- return elements ? jQuery.makeArray( elements ) : this;
- } )
- .filter( function() {
- var type = this.type;
-
- // Use .is( ":disabled" ) so that fieldset[disabled] works
- return this.name && !jQuery( this ).is( ":disabled" ) &&
- rsubmittable.test( this.nodeName ) && !rsubmitterTypes.test( type ) &&
- ( this.checked || !rcheckableType.test( type ) );
- } )
- .map( function( i, elem ) {
- var val = jQuery( this ).val();
-
- if ( val == null ) {
- return null;
- }
-
- if ( Array.isArray( val ) ) {
- return jQuery.map( val, function( val ) {
- return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) };
- } );
- }
-
- return { name: elem.name, value: val.replace( rCRLF, "\r\n" ) };
- } ).get();
- }
-} );
-
-
-var
- r20 = /%20/g,
- rhash = /#.*$/,
- rantiCache = /([?&])_=[^&]*/,
- rheaders = /^(.*?):[ \t]*([^\r\n]*)$/mg,
-
- // #7653, #8125, #8152: local protocol detection
- rlocalProtocol = /^(?:about|app|app-storage|.+-extension|file|res|widget):$/,
- rnoContent = /^(?:GET|HEAD)$/,
- rprotocol = /^\/\//,
-
- /* Prefilters
- * 1) They are useful to introduce custom dataTypes (see ajax/jsonp.js for an example)
- * 2) These are called:
- * - BEFORE asking for a transport
- * - AFTER param serialization (s.data is a string if s.processData is true)
- * 3) key is the dataType
- * 4) the catchall symbol "*" can be used
- * 5) execution will start with transport dataType and THEN continue down to "*" if needed
- */
- prefilters = {},
-
- /* Transports bindings
- * 1) key is the dataType
- * 2) the catchall symbol "*" can be used
- * 3) selection will start with transport dataType and THEN go to "*" if needed
- */
- transports = {},
-
- // Avoid comment-prolog char sequence (#10098); must appease lint and evade compression
- allTypes = "*/".concat( "*" ),
-
- // Anchor tag for parsing the document origin
- originAnchor = document.createElement( "a" );
- originAnchor.href = location.href;
-
-// Base "constructor" for jQuery.ajaxPrefilter and jQuery.ajaxTransport
-function addToPrefiltersOrTransports( structure ) {
-
- // dataTypeExpression is optional and defaults to "*"
- return function( dataTypeExpression, func ) {
-
- if ( typeof dataTypeExpression !== "string" ) {
- func = dataTypeExpression;
- dataTypeExpression = "*";
- }
-
- var dataType,
- i = 0,
- dataTypes = dataTypeExpression.toLowerCase().match( rnothtmlwhite ) || [];
-
- if ( jQuery.isFunction( func ) ) {
-
- // For each dataType in the dataTypeExpression
- while ( ( dataType = dataTypes[ i++ ] ) ) {
-
- // Prepend if requested
- if ( dataType[ 0 ] === "+" ) {
- dataType = dataType.slice( 1 ) || "*";
- ( structure[ dataType ] = structure[ dataType ] || [] ).unshift( func );
-
- // Otherwise append
- } else {
- ( structure[ dataType ] = structure[ dataType ] || [] ).push( func );
- }
- }
- }
- };
-}
-
-// Base inspection function for prefilters and transports
-function inspectPrefiltersOrTransports( structure, options, originalOptions, jqXHR ) {
-
- var inspected = {},
- seekingTransport = ( structure === transports );
-
- function inspect( dataType ) {
- var selected;
- inspected[ dataType ] = true;
- jQuery.each( structure[ dataType ] || [], function( _, prefilterOrFactory ) {
- var dataTypeOrTransport = prefilterOrFactory( options, originalOptions, jqXHR );
- if ( typeof dataTypeOrTransport === "string" &&
- !seekingTransport && !inspected[ dataTypeOrTransport ] ) {
-
- options.dataTypes.unshift( dataTypeOrTransport );
- inspect( dataTypeOrTransport );
- return false;
- } else if ( seekingTransport ) {
- return !( selected = dataTypeOrTransport );
- }
- } );
- return selected;
- }
-
- return inspect( options.dataTypes[ 0 ] ) || !inspected[ "*" ] && inspect( "*" );
-}
-
-// A special extend for ajax options
-// that takes "flat" options (not to be deep extended)
-// Fixes #9887
-function ajaxExtend( target, src ) {
- var key, deep,
- flatOptions = jQuery.ajaxSettings.flatOptions || {};
-
- for ( key in src ) {
- if ( src[ key ] !== undefined ) {
- ( flatOptions[ key ] ? target : ( deep || ( deep = {} ) ) )[ key ] = src[ key ];
- }
- }
- if ( deep ) {
- jQuery.extend( true, target, deep );
- }
-
- return target;
-}
-
-/* Handles responses to an ajax request:
- * - finds the right dataType (mediates between content-type and expected dataType)
- * - returns the corresponding response
- */
-function ajaxHandleResponses( s, jqXHR, responses ) {
-
- var ct, type, finalDataType, firstDataType,
- contents = s.contents,
- dataTypes = s.dataTypes;
-
- // Remove auto dataType and get content-type in the process
- while ( dataTypes[ 0 ] === "*" ) {
- dataTypes.shift();
- if ( ct === undefined ) {
- ct = s.mimeType || jqXHR.getResponseHeader( "Content-Type" );
- }
- }
-
- // Check if we're dealing with a known content-type
- if ( ct ) {
- for ( type in contents ) {
- if ( contents[ type ] && contents[ type ].test( ct ) ) {
- dataTypes.unshift( type );
- break;
- }
- }
- }
-
- // Check to see if we have a response for the expected dataType
- if ( dataTypes[ 0 ] in responses ) {
- finalDataType = dataTypes[ 0 ];
- } else {
-
- // Try convertible dataTypes
- for ( type in responses ) {
- if ( !dataTypes[ 0 ] || s.converters[ type + " " + dataTypes[ 0 ] ] ) {
- finalDataType = type;
- break;
- }
- if ( !firstDataType ) {
- firstDataType = type;
- }
- }
-
- // Or just use first one
- finalDataType = finalDataType || firstDataType;
- }
-
- // If we found a dataType
- // We add the dataType to the list if needed
- // and return the corresponding response
- if ( finalDataType ) {
- if ( finalDataType !== dataTypes[ 0 ] ) {
- dataTypes.unshift( finalDataType );
- }
- return responses[ finalDataType ];
- }
-}
-
-/* Chain conversions given the request and the original response
- * Also sets the responseXXX fields on the jqXHR instance
- */
-function ajaxConvert( s, response, jqXHR, isSuccess ) {
- var conv2, current, conv, tmp, prev,
- converters = {},
-
- // Work with a copy of dataTypes in case we need to modify it for conversion
- dataTypes = s.dataTypes.slice();
-
- // Create converters map with lowercased keys
- if ( dataTypes[ 1 ] ) {
- for ( conv in s.converters ) {
- converters[ conv.toLowerCase() ] = s.converters[ conv ];
- }
- }
-
- current = dataTypes.shift();
-
- // Convert to each sequential dataType
- while ( current ) {
-
- if ( s.responseFields[ current ] ) {
- jqXHR[ s.responseFields[ current ] ] = response;
- }
-
- // Apply the dataFilter if provided
- if ( !prev && isSuccess && s.dataFilter ) {
- response = s.dataFilter( response, s.dataType );
- }
-
- prev = current;
- current = dataTypes.shift();
-
- if ( current ) {
-
- // There's only work to do if current dataType is non-auto
- if ( current === "*" ) {
-
- current = prev;
-
- // Convert response if prev dataType is non-auto and differs from current
- } else if ( prev !== "*" && prev !== current ) {
-
- // Seek a direct converter
- conv = converters[ prev + " " + current ] || converters[ "* " + current ];
-
- // If none found, seek a pair
- if ( !conv ) {
- for ( conv2 in converters ) {
-
- // If conv2 outputs current
- tmp = conv2.split( " " );
- if ( tmp[ 1 ] === current ) {
-
- // If prev can be converted to accepted input
- conv = converters[ prev + " " + tmp[ 0 ] ] ||
- converters[ "* " + tmp[ 0 ] ];
- if ( conv ) {
-
- // Condense equivalence converters
- if ( conv === true ) {
- conv = converters[ conv2 ];
-
- // Otherwise, insert the intermediate dataType
- } else if ( converters[ conv2 ] !== true ) {
- current = tmp[ 0 ];
- dataTypes.unshift( tmp[ 1 ] );
- }
- break;
- }
- }
- }
- }
-
- // Apply converter (if not an equivalence)
- if ( conv !== true ) {
-
- // Unless errors are allowed to bubble, catch and return them
- if ( conv && s.throws ) {
- response = conv( response );
- } else {
- try {
- response = conv( response );
- } catch ( e ) {
- return {
- state: "parsererror",
- error: conv ? e : "No conversion from " + prev + " to " + current
- };
- }
- }
- }
- }
- }
- }
-
- return { state: "success", data: response };
-}
-
-jQuery.extend( {
-
- // Counter for holding the number of active queries
- active: 0,
-
- // Last-Modified header cache for next request
- lastModified: {},
- etag: {},
-
- ajaxSettings: {
- url: location.href,
- type: "GET",
- isLocal: rlocalProtocol.test( location.protocol ),
- global: true,
- processData: true,
- async: true,
- contentType: "application/x-www-form-urlencoded; charset=UTF-8",
-
- /*
- timeout: 0,
- data: null,
- dataType: null,
- username: null,
- password: null,
- cache: null,
- throws: false,
- traditional: false,
- headers: {},
- */
-
- accepts: {
- "*": allTypes,
- text: "text/plain",
- html: "text/html",
- xml: "application/xml, text/xml",
- json: "application/json, text/javascript"
- },
-
- contents: {
- xml: /\bxml\b/,
- html: /\bhtml/,
- json: /\bjson\b/
- },
-
- responseFields: {
- xml: "responseXML",
- text: "responseText",
- json: "responseJSON"
- },
-
- // Data converters
- // Keys separate source (or catchall "*") and destination types with a single space
- converters: {
-
- // Convert anything to text
- "* text": String,
-
- // Text to html (true = no transformation)
- "text html": true,
-
- // Evaluate text as a json expression
- "text json": JSON.parse,
-
- // Parse text as xml
- "text xml": jQuery.parseXML
- },
-
- // For options that shouldn't be deep extended:
- // you can add your own custom options here if
- // and when you create one that shouldn't be
- // deep extended (see ajaxExtend)
- flatOptions: {
- url: true,
- context: true
- }
- },
-
- // Creates a full fledged settings object into target
- // with both ajaxSettings and settings fields.
- // If target is omitted, writes into ajaxSettings.
- ajaxSetup: function( target, settings ) {
- return settings ?
-
- // Building a settings object
- ajaxExtend( ajaxExtend( target, jQuery.ajaxSettings ), settings ) :
-
- // Extending ajaxSettings
- ajaxExtend( jQuery.ajaxSettings, target );
- },
-
- ajaxPrefilter: addToPrefiltersOrTransports( prefilters ),
- ajaxTransport: addToPrefiltersOrTransports( transports ),
-
- // Main method
- ajax: function( url, options ) {
-
- // If url is an object, simulate pre-1.5 signature
- if ( typeof url === "object" ) {
- options = url;
- url = undefined;
- }
-
- // Force options to be an object
- options = options || {};
-
- var transport,
-
- // URL without anti-cache param
- cacheURL,
-
- // Response headers
- responseHeadersString,
- responseHeaders,
-
- // timeout handle
- timeoutTimer,
-
- // Url cleanup var
- urlAnchor,
-
- // Request state (becomes false upon send and true upon completion)
- completed,
-
- // To know if global events are to be dispatched
- fireGlobals,
-
- // Loop variable
- i,
-
- // uncached part of the url
- uncached,
-
- // Create the final options object
- s = jQuery.ajaxSetup( {}, options ),
-
- // Callbacks context
- callbackContext = s.context || s,
-
- // Context for global events is callbackContext if it is a DOM node or jQuery collection
- globalEventContext = s.context &&
- ( callbackContext.nodeType || callbackContext.jquery ) ?
- jQuery( callbackContext ) :
- jQuery.event,
-
- // Deferreds
- deferred = jQuery.Deferred(),
- completeDeferred = jQuery.Callbacks( "once memory" ),
-
- // Status-dependent callbacks
- statusCode = s.statusCode || {},
-
- // Headers (they are sent all at once)
- requestHeaders = {},
- requestHeadersNames = {},
-
- // Default abort message
- strAbort = "canceled",
-
- // Fake xhr
- jqXHR = {
- readyState: 0,
-
- // Builds headers hashtable if needed
- getResponseHeader: function( key ) {
- var match;
- if ( completed ) {
- if ( !responseHeaders ) {
- responseHeaders = {};
- while ( ( match = rheaders.exec( responseHeadersString ) ) ) {
- responseHeaders[ match[ 1 ].toLowerCase() ] = match[ 2 ];
- }
- }
- match = responseHeaders[ key.toLowerCase() ];
- }
- return match == null ? null : match;
- },
-
- // Raw string
- getAllResponseHeaders: function() {
- return completed ? responseHeadersString : null;
- },
-
- // Caches the header
- setRequestHeader: function( name, value ) {
- if ( completed == null ) {
- name = requestHeadersNames[ name.toLowerCase() ] =
- requestHeadersNames[ name.toLowerCase() ] || name;
- requestHeaders[ name ] = value;
- }
- return this;
- },
-
- // Overrides response content-type header
- overrideMimeType: function( type ) {
- if ( completed == null ) {
- s.mimeType = type;
- }
- return this;
- },
-
- // Status-dependent callbacks
- statusCode: function( map ) {
- var code;
- if ( map ) {
- if ( completed ) {
-
- // Execute the appropriate callbacks
- jqXHR.always( map[ jqXHR.status ] );
- } else {
-
- // Lazy-add the new callbacks in a way that preserves old ones
- for ( code in map ) {
- statusCode[ code ] = [ statusCode[ code ], map[ code ] ];
- }
- }
- }
- return this;
- },
-
- // Cancel the request
- abort: function( statusText ) {
- var finalText = statusText || strAbort;
- if ( transport ) {
- transport.abort( finalText );
- }
- done( 0, finalText );
- return this;
- }
- };
-
- // Attach deferreds
- deferred.promise( jqXHR );
-
- // Add protocol if not provided (prefilters might expect it)
- // Handle falsy url in the settings object (#10093: consistency with old signature)
- // We also use the url parameter if available
- s.url = ( ( url || s.url || location.href ) + "" )
- .replace( rprotocol, location.protocol + "//" );
-
- // Alias method option to type as per ticket #12004
- s.type = options.method || options.type || s.method || s.type;
-
- // Extract dataTypes list
- s.dataTypes = ( s.dataType || "*" ).toLowerCase().match( rnothtmlwhite ) || [ "" ];
-
- // A cross-domain request is in order when the origin doesn't match the current origin.
- if ( s.crossDomain == null ) {
- urlAnchor = document.createElement( "a" );
-
- // Support: IE <=8 - 11, Edge 12 - 13
- // IE throws exception on accessing the href property if url is malformed,
- // e.g. https://example.com:80x/
- try {
- urlAnchor.href = s.url;
-
- // Support: IE <=8 - 11 only
- // Anchor's host property isn't correctly set when s.url is relative
- urlAnchor.href = urlAnchor.href;
- s.crossDomain = originAnchor.protocol + "//" + originAnchor.host !==
- urlAnchor.protocol + "//" + urlAnchor.host;
- } catch ( e ) {
-
- // If there is an error parsing the URL, assume it is crossDomain,
- // it can be rejected by the transport if it is invalid
- s.crossDomain = true;
- }
- }
-
- // Convert data if not already a string
- if ( s.data && s.processData && typeof s.data !== "string" ) {
- s.data = jQuery.param( s.data, s.traditional );
- }
-
- // Apply prefilters
- inspectPrefiltersOrTransports( prefilters, s, options, jqXHR );
-
- // If request was aborted inside a prefilter, stop there
- if ( completed ) {
- return jqXHR;
- }
-
- // We can fire global events as of now if asked to
- // Don't fire events if jQuery.event is undefined in an AMD-usage scenario (#15118)
- fireGlobals = jQuery.event && s.global;
-
- // Watch for a new set of requests
- if ( fireGlobals && jQuery.active++ === 0 ) {
- jQuery.event.trigger( "ajaxStart" );
- }
-
- // Uppercase the type
- s.type = s.type.toUpperCase();
-
- // Determine if request has content
- s.hasContent = !rnoContent.test( s.type );
-
- // Save the URL in case we're toying with the If-Modified-Since
- // and/or If-None-Match header later on
- // Remove hash to simplify url manipulation
- cacheURL = s.url.replace( rhash, "" );
-
- // More options handling for requests with no content
- if ( !s.hasContent ) {
-
- // Remember the hash so we can put it back
- uncached = s.url.slice( cacheURL.length );
-
- // If data is available, append data to url
- if ( s.data ) {
- cacheURL += ( rquery.test( cacheURL ) ? "&" : "?" ) + s.data;
-
- // #9682: remove data so that it's not used in an eventual retry
- delete s.data;
- }
-
- // Add or update anti-cache param if needed
- if ( s.cache === false ) {
- cacheURL = cacheURL.replace( rantiCache, "$1" );
- uncached = ( rquery.test( cacheURL ) ? "&" : "?" ) + "_=" + ( nonce++ ) + uncached;
- }
-
- // Put hash and anti-cache on the URL that will be requested (gh-1732)
- s.url = cacheURL + uncached;
-
- // Change '%20' to '+' if this is encoded form body content (gh-2658)
- } else if ( s.data && s.processData &&
- ( s.contentType || "" ).indexOf( "application/x-www-form-urlencoded" ) === 0 ) {
- s.data = s.data.replace( r20, "+" );
- }
-
- // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode.
- if ( s.ifModified ) {
- if ( jQuery.lastModified[ cacheURL ] ) {
- jqXHR.setRequestHeader( "If-Modified-Since", jQuery.lastModified[ cacheURL ] );
- }
- if ( jQuery.etag[ cacheURL ] ) {
- jqXHR.setRequestHeader( "If-None-Match", jQuery.etag[ cacheURL ] );
- }
- }
-
- // Set the correct header, if data is being sent
- if ( s.data && s.hasContent && s.contentType !== false || options.contentType ) {
- jqXHR.setRequestHeader( "Content-Type", s.contentType );
- }
-
- // Set the Accepts header for the server, depending on the dataType
- jqXHR.setRequestHeader(
- "Accept",
- s.dataTypes[ 0 ] && s.accepts[ s.dataTypes[ 0 ] ] ?
- s.accepts[ s.dataTypes[ 0 ] ] +
- ( s.dataTypes[ 0 ] !== "*" ? ", " + allTypes + "; q=0.01" : "" ) :
- s.accepts[ "*" ]
- );
-
- // Check for headers option
- for ( i in s.headers ) {
- jqXHR.setRequestHeader( i, s.headers[ i ] );
- }
-
- // Allow custom headers/mimetypes and early abort
- if ( s.beforeSend &&
- ( s.beforeSend.call( callbackContext, jqXHR, s ) === false || completed ) ) {
-
- // Abort if not done already and return
- return jqXHR.abort();
- }
-
- // Aborting is no longer a cancellation
- strAbort = "abort";
-
- // Install callbacks on deferreds
- completeDeferred.add( s.complete );
- jqXHR.done( s.success );
- jqXHR.fail( s.error );
-
- // Get transport
- transport = inspectPrefiltersOrTransports( transports, s, options, jqXHR );
-
- // If no transport, we auto-abort
- if ( !transport ) {
- done( -1, "No Transport" );
- } else {
- jqXHR.readyState = 1;
-
- // Send global event
- if ( fireGlobals ) {
- globalEventContext.trigger( "ajaxSend", [ jqXHR, s ] );
- }
-
- // If request was aborted inside ajaxSend, stop there
- if ( completed ) {
- return jqXHR;
- }
-
- // Timeout
- if ( s.async && s.timeout > 0 ) {
- timeoutTimer = window.setTimeout( function() {
- jqXHR.abort( "timeout" );
- }, s.timeout );
- }
-
- try {
- completed = false;
- transport.send( requestHeaders, done );
- } catch ( e ) {
-
- // Rethrow post-completion exceptions
- if ( completed ) {
- throw e;
- }
-
- // Propagate others as results
- done( -1, e );
- }
- }
-
- // Callback for when everything is done
- function done( status, nativeStatusText, responses, headers ) {
- var isSuccess, success, error, response, modified,
- statusText = nativeStatusText;
-
- // Ignore repeat invocations
- if ( completed ) {
- return;
- }
-
- completed = true;
-
- // Clear timeout if it exists
- if ( timeoutTimer ) {
- window.clearTimeout( timeoutTimer );
- }
-
- // Dereference transport for early garbage collection
- // (no matter how long the jqXHR object will be used)
- transport = undefined;
-
- // Cache response headers
- responseHeadersString = headers || "";
-
- // Set readyState
- jqXHR.readyState = status > 0 ? 4 : 0;
-
- // Determine if successful
- isSuccess = status >= 200 && status < 300 || status === 304;
-
- // Get response data
- if ( responses ) {
- response = ajaxHandleResponses( s, jqXHR, responses );
- }
-
- // Convert no matter what (that way responseXXX fields are always set)
- response = ajaxConvert( s, response, jqXHR, isSuccess );
-
- // If successful, handle type chaining
- if ( isSuccess ) {
-
- // Set the If-Modified-Since and/or If-None-Match header, if in ifModified mode.
- if ( s.ifModified ) {
- modified = jqXHR.getResponseHeader( "Last-Modified" );
- if ( modified ) {
- jQuery.lastModified[ cacheURL ] = modified;
- }
- modified = jqXHR.getResponseHeader( "etag" );
- if ( modified ) {
- jQuery.etag[ cacheURL ] = modified;
- }
- }
-
- // if no content
- if ( status === 204 || s.type === "HEAD" ) {
- statusText = "nocontent";
-
- // if not modified
- } else if ( status === 304 ) {
- statusText = "notmodified";
-
- // If we have data, let's convert it
- } else {
- statusText = response.state;
- success = response.data;
- error = response.error;
- isSuccess = !error;
- }
- } else {
-
- // Extract error from statusText and normalize for non-aborts
- error = statusText;
- if ( status || !statusText ) {
- statusText = "error";
- if ( status < 0 ) {
- status = 0;
- }
- }
- }
-
- // Set data for the fake xhr object
- jqXHR.status = status;
- jqXHR.statusText = ( nativeStatusText || statusText ) + "";
-
- // Success/Error
- if ( isSuccess ) {
- deferred.resolveWith( callbackContext, [ success, statusText, jqXHR ] );
- } else {
- deferred.rejectWith( callbackContext, [ jqXHR, statusText, error ] );
- }
-
- // Status-dependent callbacks
- jqXHR.statusCode( statusCode );
- statusCode = undefined;
-
- if ( fireGlobals ) {
- globalEventContext.trigger( isSuccess ? "ajaxSuccess" : "ajaxError",
- [ jqXHR, s, isSuccess ? success : error ] );
- }
-
- // Complete
- completeDeferred.fireWith( callbackContext, [ jqXHR, statusText ] );
-
- if ( fireGlobals ) {
- globalEventContext.trigger( "ajaxComplete", [ jqXHR, s ] );
-
- // Handle the global AJAX counter
- if ( !( --jQuery.active ) ) {
- jQuery.event.trigger( "ajaxStop" );
- }
- }
- }
-
- return jqXHR;
- },
-
- getJSON: function( url, data, callback ) {
- return jQuery.get( url, data, callback, "json" );
- },
-
- getScript: function( url, callback ) {
- return jQuery.get( url, undefined, callback, "script" );
- }
-} );
-
-jQuery.each( [ "get", "post" ], function( i, method ) {
- jQuery[ method ] = function( url, data, callback, type ) {
-
- // Shift arguments if data argument was omitted
- if ( jQuery.isFunction( data ) ) {
- type = type || callback;
- callback = data;
- data = undefined;
- }
-
- // The url can be an options object (which then must have .url)
- return jQuery.ajax( jQuery.extend( {
- url: url,
- type: method,
- dataType: type,
- data: data,
- success: callback
- }, jQuery.isPlainObject( url ) && url ) );
- };
-} );
-
-
-jQuery._evalUrl = function( url ) {
- return jQuery.ajax( {
- url: url,
-
- // Make this explicit, since user can override this through ajaxSetup (#11264)
- type: "GET",
- dataType: "script",
- cache: true,
- async: false,
- global: false,
- "throws": true
- } );
-};
-
-
-jQuery.fn.extend( {
- wrapAll: function( html ) {
- var wrap;
-
- if ( this[ 0 ] ) {
- if ( jQuery.isFunction( html ) ) {
- html = html.call( this[ 0 ] );
- }
-
- // The elements to wrap the target around
- wrap = jQuery( html, this[ 0 ].ownerDocument ).eq( 0 ).clone( true );
-
- if ( this[ 0 ].parentNode ) {
- wrap.insertBefore( this[ 0 ] );
- }
-
- wrap.map( function() {
- var elem = this;
-
- while ( elem.firstElementChild ) {
- elem = elem.firstElementChild;
- }
-
- return elem;
- } ).append( this );
- }
-
- return this;
- },
-
- wrapInner: function( html ) {
- if ( jQuery.isFunction( html ) ) {
- return this.each( function( i ) {
- jQuery( this ).wrapInner( html.call( this, i ) );
- } );
- }
-
- return this.each( function() {
- var self = jQuery( this ),
- contents = self.contents();
-
- if ( contents.length ) {
- contents.wrapAll( html );
-
- } else {
- self.append( html );
- }
- } );
- },
-
- wrap: function( html ) {
- var isFunction = jQuery.isFunction( html );
-
- return this.each( function( i ) {
- jQuery( this ).wrapAll( isFunction ? html.call( this, i ) : html );
- } );
- },
-
- unwrap: function( selector ) {
- this.parent( selector ).not( "body" ).each( function() {
- jQuery( this ).replaceWith( this.childNodes );
- } );
- return this;
- }
-} );
-
-
-jQuery.expr.pseudos.hidden = function( elem ) {
- return !jQuery.expr.pseudos.visible( elem );
-};
-jQuery.expr.pseudos.visible = function( elem ) {
- return !!( elem.offsetWidth || elem.offsetHeight || elem.getClientRects().length );
-};
-
-
-
-
-jQuery.ajaxSettings.xhr = function() {
- try {
- return new window.XMLHttpRequest();
- } catch ( e ) {}
-};
-
-var xhrSuccessStatus = {
-
- // File protocol always yields status code 0, assume 200
- 0: 200,
-
- // Support: IE <=9 only
- // #1450: sometimes IE returns 1223 when it should be 204
- 1223: 204
- },
- xhrSupported = jQuery.ajaxSettings.xhr();
-
-support.cors = !!xhrSupported && ( "withCredentials" in xhrSupported );
-support.ajax = xhrSupported = !!xhrSupported;
-
-jQuery.ajaxTransport( function( options ) {
- var callback, errorCallback;
-
- // Cross domain only allowed if supported through XMLHttpRequest
- if ( support.cors || xhrSupported && !options.crossDomain ) {
- return {
- send: function( headers, complete ) {
- var i,
- xhr = options.xhr();
-
- xhr.open(
- options.type,
- options.url,
- options.async,
- options.username,
- options.password
- );
-
- // Apply custom fields if provided
- if ( options.xhrFields ) {
- for ( i in options.xhrFields ) {
- xhr[ i ] = options.xhrFields[ i ];
- }
- }
-
- // Override mime type if needed
- if ( options.mimeType && xhr.overrideMimeType ) {
- xhr.overrideMimeType( options.mimeType );
- }
-
- // X-Requested-With header
- // For cross-domain requests, seeing as conditions for a preflight are
- // akin to a jigsaw puzzle, we simply never set it to be sure.
- // (it can always be set on a per-request basis or even using ajaxSetup)
- // For same-domain requests, won't change header if already provided.
- if ( !options.crossDomain && !headers[ "X-Requested-With" ] ) {
- headers[ "X-Requested-With" ] = "XMLHttpRequest";
- }
-
- // Set headers
- for ( i in headers ) {
- xhr.setRequestHeader( i, headers[ i ] );
- }
-
- // Callback
- callback = function( type ) {
- return function() {
- if ( callback ) {
- callback = errorCallback = xhr.onload =
- xhr.onerror = xhr.onabort = xhr.onreadystatechange = null;
-
- if ( type === "abort" ) {
- xhr.abort();
- } else if ( type === "error" ) {
-
- // Support: IE <=9 only
- // On a manual native abort, IE9 throws
- // errors on any property access that is not readyState
- if ( typeof xhr.status !== "number" ) {
- complete( 0, "error" );
- } else {
- complete(
-
- // File: protocol always yields status 0; see #8605, #14207
- xhr.status,
- xhr.statusText
- );
- }
- } else {
- complete(
- xhrSuccessStatus[ xhr.status ] || xhr.status,
- xhr.statusText,
-
- // Support: IE <=9 only
- // IE9 has no XHR2 but throws on binary (trac-11426)
- // For XHR2 non-text, let the caller handle it (gh-2498)
- ( xhr.responseType || "text" ) !== "text" ||
- typeof xhr.responseText !== "string" ?
- { binary: xhr.response } :
- { text: xhr.responseText },
- xhr.getAllResponseHeaders()
- );
- }
- }
- };
- };
-
- // Listen to events
- xhr.onload = callback();
- errorCallback = xhr.onerror = callback( "error" );
-
- // Support: IE 9 only
- // Use onreadystatechange to replace onabort
- // to handle uncaught aborts
- if ( xhr.onabort !== undefined ) {
- xhr.onabort = errorCallback;
- } else {
- xhr.onreadystatechange = function() {
-
- // Check readyState before timeout as it changes
- if ( xhr.readyState === 4 ) {
-
- // Allow onerror to be called first,
- // but that will not handle a native abort
- // Also, save errorCallback to a variable
- // as xhr.onerror cannot be accessed
- window.setTimeout( function() {
- if ( callback ) {
- errorCallback();
- }
- } );
- }
- };
- }
-
- // Create the abort callback
- callback = callback( "abort" );
-
- try {
-
- // Do send the request (this may raise an exception)
- xhr.send( options.hasContent && options.data || null );
- } catch ( e ) {
-
- // #14683: Only rethrow if this hasn't been notified as an error yet
- if ( callback ) {
- throw e;
- }
- }
- },
-
- abort: function() {
- if ( callback ) {
- callback();
- }
- }
- };
- }
-} );
-
-
-
-
-// Prevent auto-execution of scripts when no explicit dataType was provided (See gh-2432)
-jQuery.ajaxPrefilter( function( s ) {
- if ( s.crossDomain ) {
- s.contents.script = false;
- }
-} );
-
-// Install script dataType
-jQuery.ajaxSetup( {
- accepts: {
- script: "text/javascript, application/javascript, " +
- "application/ecmascript, application/x-ecmascript"
- },
- contents: {
- script: /\b(?:java|ecma)script\b/
- },
- converters: {
- "text script": function( text ) {
- jQuery.globalEval( text );
- return text;
- }
- }
-} );
-
-// Handle cache's special case and crossDomain
-jQuery.ajaxPrefilter( "script", function( s ) {
- if ( s.cache === undefined ) {
- s.cache = false;
- }
- if ( s.crossDomain ) {
- s.type = "GET";
- }
-} );
-
-// Bind script tag hack transport
-jQuery.ajaxTransport( "script", function( s ) {
-
- // This transport only deals with cross domain requests
- if ( s.crossDomain ) {
- var script, callback;
- return {
- send: function( _, complete ) {
- script = jQuery( "
-{%- endblock %}
-
-
-{% block footer %}
-{{ super() }}
-
-
-
-
-
-
-
-`_\ ),
-we see that the input is named '0' and the output is named '186'. These are just numeric ids assigned by PyTorch.
-We will need to mark these as images.
-
-So let's create a small python file and call it ``onnx_to_coreml.py``. This can be created by using the touch command and edited with your favorite editor to add the following lines of code.
-
-.. code-block:: python
-
- import sys
- from onnx import onnx_pb
- from onnx_coreml import convert
-
- model_in = sys.argv[1]
- model_out = sys.argv[2]
-
- model_file = open(model_in, 'rb')
- model_proto = onnx_pb.ModelProto()
- model_proto.ParseFromString(model_file.read())
- coreml_model = convert(model_proto, image_input_names=['0'], image_output_names=['186'])
- coreml_model.save(model_out)
-
-
-we now run it:
-
-.. code-block:: bash
-
- python onnx_to_coreml.py ./saved_models/candy.onnx ./saved_models/candy.mlmodel
- python onnx_to_coreml.py ./saved_models/udnie.onnx ./saved_models/udnie.mlmodel
- python onnx_to_coreml.py ./saved_models/rain_princess.onnx ./saved_models/rain_princess.mlmodel
- python onnx_to_coreml.py ./saved_models/mosaic.onnx ./saved_models/mosaic.mlmodel
-
-
-Now, there should be 4 CoreML models in your ``saved_models`` directory: ``candy.mlmodel``\ , ``mosaic.mlmodel``\ , ``rain_princess.mlmodel`` and ``udnie.mlmodel``.
-
-Run the CoreML models in a style transfer iOS App
--------------------------------------------------
-
-This repository (i.e. the one you're currently reading the README.md of) contains an iOS app able to run CoreML style transfer models on a live camera stream from your phone camera. Let's clone the repository:
-
-.. code-block:: bash
-
- git clone https://github.com/onnx/tutorials
-
-
-and open the ``tutorials/examples/CoreML/ONNXLive/ONNXLive.xcodeproj`` project in XCode.
-We recommend using XCode 9.3 and an iPhone X. There might be issues running on older devices or XCode versions.
-
-In the ``Models/`` folder, the project contains some .mlmodel files. We're going to replace them with the models we just created.
-
-You then run the app on your iPhone and you are all set. Tapping on the screen switches through the models.
-
-Conclusion
-----------
-
-We hope this tutorial gave you an overview of what ONNX is about and how you can use it to convert neural networks
-between frameworks, in this case neural style transfer models moving from PyTorch to CoreML.
-
-Feel free to experiment with these steps and test them on your own models.
-Please let us know if you hit any issues or want to give feedback. We'd like to hear what you think.
+ `_)
+(`link `_)
and ``torch::autograd::grad``
-(`link `_)
+(`link `_)
for more information on how to use them.
Using custom autograd function in C++
@@ -394,9 +394,9 @@ C++ using the following table:
+--------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Python | C++ |
+================================+========================================================================================================================================================================+
-| ``torch.autograd.backward`` | ``torch::autograd::backward`` (`link `_) |
+| ``torch.autograd.backward`` | ``torch::autograd::backward`` (`link `_) |
+--------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-| ``torch.autograd.grad`` | ``torch::autograd::grad`` (`link `_) |
+| ``torch.autograd.grad`` | ``torch::autograd::grad`` (`link `_) |
+--------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``torch.Tensor.detach`` | ``torch::Tensor::detach`` (`link `_) |
+--------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
diff --git a/advanced_source/cpp_cuda_graphs.rst b/advanced_source/cpp_cuda_graphs.rst
deleted file mode 100644
index 494d6426d47..00000000000
--- a/advanced_source/cpp_cuda_graphs.rst
+++ /dev/null
@@ -1,193 +0,0 @@
-Using CUDA Graphs in PyTorch C++ API
-====================================
-
-.. note::
- |edit| View and edit this tutorial in `GitHub `__. The full source code is available on `GitHub `__.
-
-Prerequisites:
-
-- `Using the PyTorch C++ Frontend <../advanced_source/cpp_frontend.html>`__
-- `CUDA semantics `__
-- Pytorch 2.0 or later
-- CUDA 11 or later
-
-NVIDIA’s CUDA Graphs have been a part of CUDA Toolkit library since the
-release of `version 10 `_.
-The C++ frontend is mostly utilized in production and deployment applications which
-are important parts of PyTorch use cases. Since `the first appearance
-`_.
-The usage of CUDA Graphs in LibTorch (C++ Frontend) is very similar to its
-`Python counterpart `_
-but with some differences in syntax and functionality.
-
-Getting Started
----------------
-
-The main training loop consists of the several steps and depicted in the
-following code chunk:
-
-.. code-block:: cpp
-
- for (auto& batch : data_loader) {
- auto data = batch.data.to(device);
- auto targets = batch.target.to(device);
- optimizer.zero_grad();
- auto output = model.forward(data);
- auto loss = torch::nll_loss(output, targets);
- loss.backward();
- optimizer.step();
- }
-
-The example above includes a forward pass, a backward pass, and weight updates.
-
-In this tutorial, we will be applying CUDA Graph on all the compute steps through the whole-network
-graph capture. But before doing so, we need to slightly modify the source code. What we need
-to do is preallocate tensors for reusing them in the main training loop. Here is an example
-implementation:
-
-.. code-block:: cpp
-
- torch::TensorOptions FloatCUDA =
- torch::TensorOptions(device).dtype(torch::kFloat);
- torch::TensorOptions LongCUDA =
- torch::TensorOptions(device).dtype(torch::kLong);
-
- torch::Tensor data = torch::zeros({kTrainBatchSize, 1, 28, 28}, FloatCUDA);
- torch::Tensor targets = torch::zeros({kTrainBatchSize}, LongCUDA);
- torch::Tensor output = torch::zeros({1}, FloatCUDA);
- torch::Tensor loss = torch::zeros({1}, FloatCUDA);
-
- for (auto& batch : data_loader) {
- data.copy_(batch.data);
- targets.copy_(batch.target);
- training_step(model, optimizer, data, targets, output, loss);
- }
-
-Where ``training_step`` simply consists of forward and backward passes with corresponding optimizer calls:
-
-.. code-block:: cpp
-
- void training_step(
- Net& model,
- torch::optim::Optimizer& optimizer,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss) {
- optimizer.zero_grad();
- output = model.forward(data);
- loss = torch::nll_loss(output, targets);
- loss.backward();
- optimizer.step();
- }
-
-PyTorch’s CUDA Graphs API is relying on Stream Capture which in our case would be used like this:
-
-.. code-block:: cpp
-
- at::cuda::CUDAGraph graph;
- at::cuda::CUDAStream captureStream = at::cuda::getStreamFromPool();
- at::cuda::setCurrentCUDAStream(captureStream);
-
- graph.capture_begin();
- training_step(model, optimizer, data, targets, output, loss);
- graph.capture_end();
-
-Before the actual graph capture, it is important to run several warm-up iterations on side stream to
-prepare CUDA cache as well as CUDA libraries (like CUBLAS and CUDNN) that will be used during
-the training:
-
-.. code-block:: cpp
-
- at::cuda::CUDAStream warmupStream = at::cuda::getStreamFromPool();
- at::cuda::setCurrentCUDAStream(warmupStream);
- for (int iter = 0; iter < num_warmup_iters; iter++) {
- training_step(model, optimizer, data, targets, output, loss);
- }
-
-After the successful graph capture, we can replace ``training_step(model, optimizer, data, targets, output, loss);``
-call via ``graph.replay();`` to do the training step.
-
-Training Results
-----------------
-
-Taking the code for a spin we can see the following output from ordinary non-graphed training:
-
-.. code-block:: shell
-
- $ time ./mnist
- Train Epoch: 1 [59584/60000] Loss: 0.3921
- Test set: Average loss: 0.2051 | Accuracy: 0.938
- Train Epoch: 2 [59584/60000] Loss: 0.1826
- Test set: Average loss: 0.1273 | Accuracy: 0.960
- Train Epoch: 3 [59584/60000] Loss: 0.1796
- Test set: Average loss: 0.1012 | Accuracy: 0.968
- Train Epoch: 4 [59584/60000] Loss: 0.1603
- Test set: Average loss: 0.0869 | Accuracy: 0.973
- Train Epoch: 5 [59584/60000] Loss: 0.2315
- Test set: Average loss: 0.0736 | Accuracy: 0.978
- Train Epoch: 6 [59584/60000] Loss: 0.0511
- Test set: Average loss: 0.0704 | Accuracy: 0.977
- Train Epoch: 7 [59584/60000] Loss: 0.0802
- Test set: Average loss: 0.0654 | Accuracy: 0.979
- Train Epoch: 8 [59584/60000] Loss: 0.0774
- Test set: Average loss: 0.0604 | Accuracy: 0.980
- Train Epoch: 9 [59584/60000] Loss: 0.0669
- Test set: Average loss: 0.0544 | Accuracy: 0.984
- Train Epoch: 10 [59584/60000] Loss: 0.0219
- Test set: Average loss: 0.0517 | Accuracy: 0.983
-
- real 0m44.287s
- user 0m44.018s
- sys 0m1.116s
-
-While the training with the CUDA Graph produces the following output:
-
-.. code-block:: shell
-
- $ time ./mnist --use-train-graph
- Train Epoch: 1 [59584/60000] Loss: 0.4092
- Test set: Average loss: 0.2037 | Accuracy: 0.938
- Train Epoch: 2 [59584/60000] Loss: 0.2039
- Test set: Average loss: 0.1274 | Accuracy: 0.961
- Train Epoch: 3 [59584/60000] Loss: 0.1779
- Test set: Average loss: 0.1017 | Accuracy: 0.968
- Train Epoch: 4 [59584/60000] Loss: 0.1559
- Test set: Average loss: 0.0871 | Accuracy: 0.972
- Train Epoch: 5 [59584/60000] Loss: 0.2240
- Test set: Average loss: 0.0735 | Accuracy: 0.977
- Train Epoch: 6 [59584/60000] Loss: 0.0520
- Test set: Average loss: 0.0710 | Accuracy: 0.978
- Train Epoch: 7 [59584/60000] Loss: 0.0935
- Test set: Average loss: 0.0666 | Accuracy: 0.979
- Train Epoch: 8 [59584/60000] Loss: 0.0744
- Test set: Average loss: 0.0603 | Accuracy: 0.981
- Train Epoch: 9 [59584/60000] Loss: 0.0762
- Test set: Average loss: 0.0547 | Accuracy: 0.983
- Train Epoch: 10 [59584/60000] Loss: 0.0207
- Test set: Average loss: 0.0525 | Accuracy: 0.983
-
- real 0m6.952s
- user 0m7.048s
- sys 0m0.619s
-
-Conclusion
-----------
-
-As we can see, just by applying a CUDA Graph on the `MNIST example
-`_ we were able to gain the performance
-by more than six times for training. This kind of large performance improvement was achievable due to
-the small model size. In case of larger models with heavy GPU usage, the CPU overhead is less impactful
-so the improvement will be smaller. Nevertheless, it is always advantageous to use CUDA Graphs to
-gain the performance of GPUs.
diff --git a/advanced_source/cpp_cuda_graphs/CMakeLists.txt b/advanced_source/cpp_cuda_graphs/CMakeLists.txt
deleted file mode 100644
index 76fc5bc6762..00000000000
--- a/advanced_source/cpp_cuda_graphs/CMakeLists.txt
+++ /dev/null
@@ -1,31 +0,0 @@
-cmake_minimum_required(VERSION 3.18 FATAL_ERROR)
-project(mnist)
-set(CMAKE_CXX_STANDARD 17)
-
-find_package(Torch REQUIRED)
-find_package(Threads REQUIRED)
-
-option(DOWNLOAD_MNIST "Download the MNIST dataset from the internet" ON)
-if (DOWNLOAD_MNIST)
- message(STATUS "Downloading MNIST dataset")
- execute_process(
- COMMAND python ${CMAKE_CURRENT_LIST_DIR}/../tools/download_mnist.py
- -d ${CMAKE_BINARY_DIR}/data
- ERROR_VARIABLE DOWNLOAD_ERROR)
- if (DOWNLOAD_ERROR)
- message(FATAL_ERROR "Error downloading MNIST dataset: ${DOWNLOAD_ERROR}")
- endif()
-endif()
-
-add_executable(mnist mnist.cpp)
-target_compile_features(mnist PUBLIC cxx_range_for)
-target_link_libraries(mnist ${TORCH_LIBRARIES} ${CMAKE_THREAD_LIBS_INIT})
-
-if (MSVC)
- file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
- add_custom_command(TARGET mnist
- POST_BUILD
- COMMAND ${CMAKE_COMMAND} -E copy_if_different
- ${TORCH_DLLS}
- $)
-endif (MSVC)
diff --git a/advanced_source/cpp_cuda_graphs/README.md b/advanced_source/cpp_cuda_graphs/README.md
deleted file mode 100644
index cbe368d1e90..00000000000
--- a/advanced_source/cpp_cuda_graphs/README.md
+++ /dev/null
@@ -1,38 +0,0 @@
-# MNIST Example with the PyTorch C++ Frontend
-
-This folder contains an example of training a computer vision model to recognize
-digits in images from the MNIST dataset, using the PyTorch C++ frontend.
-
-The entire training code is contained in `mnist.cpp`.
-
-To build the code, run the following commands from your terminal:
-
-```shell
-$ cd mnist
-$ mkdir build
-$ cd build
-$ cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch ..
-$ make
-```
-
-where `/path/to/libtorch` should be the path to the unzipped _LibTorch_
-distribution, which you can get from the [PyTorch
-homepage](https://pytorch.org/get-started/locally/).
-
-Execute the compiled binary to train the model:
-
-```shell
-$ ./mnist
-Train Epoch: 1 [59584/60000] Loss: 0.4232
-Test set: Average loss: 0.1989 | Accuracy: 0.940
-Train Epoch: 2 [59584/60000] Loss: 0.1926
-Test set: Average loss: 0.1338 | Accuracy: 0.959
-Train Epoch: 3 [59584/60000] Loss: 0.1390
-Test set: Average loss: 0.0997 | Accuracy: 0.969
-Train Epoch: 4 [59584/60000] Loss: 0.1239
-Test set: Average loss: 0.0875 | Accuracy: 0.972
-...
-```
-
-For running with CUDA Graphs add `--use-train-graph` and/or `--use-test-graph`
-for training and testing passes respectively.
diff --git a/advanced_source/cpp_cuda_graphs/mnist.cpp b/advanced_source/cpp_cuda_graphs/mnist.cpp
deleted file mode 100644
index 97c5fb80ca0..00000000000
--- a/advanced_source/cpp_cuda_graphs/mnist.cpp
+++ /dev/null
@@ -1,372 +0,0 @@
-#include
-#include
-#include
-#include
-
-#include
-#include
-#include
-#include
-#include
-
-// Where to find the MNIST dataset.
-const char* kDataRoot = "./data";
-
-// The batch size for training.
-const int64_t kTrainBatchSize = 64;
-
-// The batch size for testing.
-const int64_t kTestBatchSize = 1000;
-
-// The number of epochs to train.
-const int64_t kNumberOfEpochs = 10;
-
-// After how many batches to log a new update with the loss value.
-const int64_t kLogInterval = 10;
-
-// Model that we will be training
-struct Net : torch::nn::Module {
- Net()
- : conv1(torch::nn::Conv2dOptions(1, 10, /*kernel_size=*/5)),
- conv2(torch::nn::Conv2dOptions(10, 20, /*kernel_size=*/5)),
- fc1(320, 50),
- fc2(50, 10) {
- register_module("conv1", conv1);
- register_module("conv2", conv2);
- register_module("conv2_drop", conv2_drop);
- register_module("fc1", fc1);
- register_module("fc2", fc2);
- }
-
- torch::Tensor forward(torch::Tensor x) {
- x = torch::relu(torch::max_pool2d(conv1->forward(x), 2));
- x = torch::relu(
- torch::max_pool2d(conv2_drop->forward(conv2->forward(x)), 2));
- x = x.view({-1, 320});
- x = torch::relu(fc1->forward(x));
- x = torch::dropout(x, /*p=*/0.5, /*training=*/is_training());
- x = fc2->forward(x);
- return torch::log_softmax(x, /*dim=*/1);
- }
-
- torch::nn::Conv2d conv1;
- torch::nn::Conv2d conv2;
- torch::nn::Dropout2d conv2_drop;
- torch::nn::Linear fc1;
- torch::nn::Linear fc2;
-};
-
-void stream_sync(
- at::cuda::CUDAStream& dependency,
- at::cuda::CUDAStream& dependent) {
- at::cuda::CUDAEvent cuda_ev;
- cuda_ev.record(dependency);
- cuda_ev.block(dependent);
-}
-
-void training_step(
- Net& model,
- torch::optim::Optimizer& optimizer,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss) {
- optimizer.zero_grad();
- output = model.forward(data);
- loss = torch::nll_loss(output, targets);
- loss.backward();
- optimizer.step();
-}
-
-void capture_train_graph(
- Net& model,
- torch::optim::Optimizer& optimizer,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss,
- at::cuda::CUDAGraph& graph,
- const short num_warmup_iters = 7) {
- model.train();
-
- auto warmupStream = at::cuda::getStreamFromPool();
- auto captureStream = at::cuda::getStreamFromPool();
- auto legacyStream = at::cuda::getCurrentCUDAStream();
-
- at::cuda::setCurrentCUDAStream(warmupStream);
-
- stream_sync(legacyStream, warmupStream);
-
- for (C10_UNUSED const auto iter : c10::irange(num_warmup_iters)) {
- training_step(model, optimizer, data, targets, output, loss);
- }
-
- stream_sync(warmupStream, captureStream);
- at::cuda::setCurrentCUDAStream(captureStream);
-
- graph.capture_begin();
- training_step(model, optimizer, data, targets, output, loss);
- graph.capture_end();
-
- stream_sync(captureStream, legacyStream);
- at::cuda::setCurrentCUDAStream(legacyStream);
-}
-
-template
-void train(
- size_t epoch,
- Net& model,
- torch::Device device,
- DataLoader& data_loader,
- torch::optim::Optimizer& optimizer,
- size_t dataset_size,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss,
- at::cuda::CUDAGraph& graph,
- bool use_graph) {
- model.train();
-
- size_t batch_idx = 0;
-
- for (const auto& batch : data_loader) {
- if (batch.data.size(0) != kTrainBatchSize ||
- batch.target.size(0) != kTrainBatchSize) {
- continue;
- }
-
- data.copy_(batch.data);
- targets.copy_(batch.target);
-
- if (use_graph) {
- graph.replay();
- } else {
- training_step(model, optimizer, data, targets, output, loss);
- }
-
- if (batch_idx++ % kLogInterval == 0) {
- float train_loss = loss.item();
- std::cout << "\rTrain Epoch:" << epoch << " ["
- << batch_idx * batch.data.size(0) << "/" << dataset_size
- << "] Loss: " << train_loss;
- }
- }
-}
-
-void test_step(
- Net& model,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss) {
- output = model.forward(data);
- loss = torch::nll_loss(output, targets, {}, torch::Reduction::Sum);
-}
-
-void capture_test_graph(
- Net& model,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss,
- torch::Tensor& total_loss,
- torch::Tensor& total_correct,
- at::cuda::CUDAGraph& graph,
- const int num_warmup_iters = 7) {
- torch::NoGradGuard no_grad;
- model.eval();
-
- auto warmupStream = at::cuda::getStreamFromPool();
- auto captureStream = at::cuda::getStreamFromPool();
- auto legacyStream = at::cuda::getCurrentCUDAStream();
-
- at::cuda::setCurrentCUDAStream(warmupStream);
- stream_sync(captureStream, legacyStream);
-
- for (C10_UNUSED const auto iter : c10::irange(num_warmup_iters)) {
- test_step(model, data, targets, output, loss);
- total_loss += loss;
- total_correct += output.argmax(1).eq(targets).sum();
- }
-
- stream_sync(warmupStream, captureStream);
- at::cuda::setCurrentCUDAStream(captureStream);
-
- graph.capture_begin();
- test_step(model, data, targets, output, loss);
- graph.capture_end();
-
- stream_sync(captureStream, legacyStream);
- at::cuda::setCurrentCUDAStream(legacyStream);
-}
-
-template
-void test(
- Net& model,
- torch::Device device,
- DataLoader& data_loader,
- size_t dataset_size,
- torch::Tensor& data,
- torch::Tensor& targets,
- torch::Tensor& output,
- torch::Tensor& loss,
- torch::Tensor& total_loss,
- torch::Tensor& total_correct,
- at::cuda::CUDAGraph& graph,
- bool use_graph) {
- torch::NoGradGuard no_grad;
-
- model.eval();
- loss.zero_();
- total_loss.zero_();
- total_correct.zero_();
-
- for (const auto& batch : data_loader) {
- if (batch.data.size(0) != kTestBatchSize ||
- batch.target.size(0) != kTestBatchSize) {
- continue;
- }
- data.copy_(batch.data);
- targets.copy_(batch.target);
-
- if (use_graph) {
- graph.replay();
- } else {
- test_step(model, data, targets, output, loss);
- }
- total_loss += loss;
- total_correct += output.argmax(1).eq(targets).sum();
- }
-
- float test_loss = total_loss.item() / dataset_size;
- float test_accuracy =
- static_cast(total_correct.item()) / dataset_size;
-
- std::cout << std::endl
- << "Test set: Average loss: " << test_loss
- << " | Accuracy: " << test_accuracy << std::endl;
-}
-
-int main(int argc, char* argv[]) {
- if (!torch::cuda::is_available()) {
- std::cout << "CUDA is not available!" << std::endl;
- return -1;
- }
-
- bool use_train_graph = false;
- bool use_test_graph = false;
-
- std::vector arguments(argv + 1, argv + argc);
- for (std::string& arg : arguments) {
- if (arg == "--use-train-graph") {
- std::cout << "Using CUDA Graph for training." << std::endl;
- use_train_graph = true;
- }
- if (arg == "--use-test-graph") {
- std::cout << "Using CUDA Graph for testing." << std::endl;
- use_test_graph = true;
- }
- }
-
- torch::manual_seed(1);
- torch::cuda::manual_seed(1);
- torch::Device device(torch::kCUDA);
-
- Net model;
- model.to(device);
-
- auto train_dataset =
- torch::data::datasets::MNIST(kDataRoot)
- .map(torch::data::transforms::Normalize<>(0.1307, 0.3081))
- .map(torch::data::transforms::Stack<>());
- const size_t train_dataset_size = train_dataset.size().value();
- auto train_loader =
- torch::data::make_data_loader(
- std::move(train_dataset), kTrainBatchSize);
-
- auto test_dataset =
- torch::data::datasets::MNIST(
- kDataRoot, torch::data::datasets::MNIST::Mode::kTest)
- .map(torch::data::transforms::Normalize<>(0.1307, 0.3081))
- .map(torch::data::transforms::Stack<>());
- const size_t test_dataset_size = test_dataset.size().value();
- auto test_loader =
- torch::data::make_data_loader(std::move(test_dataset), kTestBatchSize);
-
- torch::optim::SGD optimizer(
- model.parameters(), torch::optim::SGDOptions(0.01).momentum(0.5));
-
- torch::TensorOptions FloatCUDA =
- torch::TensorOptions(device).dtype(torch::kFloat);
- torch::TensorOptions LongCUDA =
- torch::TensorOptions(device).dtype(torch::kLong);
-
- torch::Tensor train_data =
- torch::zeros({kTrainBatchSize, 1, 28, 28}, FloatCUDA);
- torch::Tensor train_targets = torch::zeros({kTrainBatchSize}, LongCUDA);
- torch::Tensor train_output = torch::zeros({1}, FloatCUDA);
- torch::Tensor train_loss = torch::zeros({1}, FloatCUDA);
-
- torch::Tensor test_data =
- torch::zeros({kTestBatchSize, 1, 28, 28}, FloatCUDA);
- torch::Tensor test_targets = torch::zeros({kTestBatchSize}, LongCUDA);
- torch::Tensor test_output = torch::zeros({1}, FloatCUDA);
- torch::Tensor test_loss = torch::zeros({1}, FloatCUDA);
- torch::Tensor test_total_loss = torch::zeros({1}, FloatCUDA);
- torch::Tensor test_total_correct = torch::zeros({1}, LongCUDA);
-
- at::cuda::CUDAGraph train_graph;
- at::cuda::CUDAGraph test_graph;
-
- capture_train_graph(
- model,
- optimizer,
- train_data,
- train_targets,
- train_output,
- train_loss,
- train_graph);
-
- capture_test_graph(
- model,
- test_data,
- test_targets,
- test_output,
- test_loss,
- test_total_loss,
- test_total_correct,
- test_graph);
-
- for (size_t epoch = 1; epoch <= kNumberOfEpochs; ++epoch) {
- train(
- epoch,
- model,
- device,
- *train_loader,
- optimizer,
- train_dataset_size,
- train_data,
- train_targets,
- train_output,
- train_loss,
- train_graph,
- use_train_graph);
- test(
- model,
- device,
- *test_loader,
- test_dataset_size,
- test_data,
- test_targets,
- test_output,
- test_loss,
- test_total_loss,
- test_total_correct,
- test_graph,
- use_test_graph);
- }
-
- std::cout << " Training/testing complete" << std::endl;
- return 0;
-}
diff --git a/advanced_source/cpp_custom_ops.rst b/advanced_source/cpp_custom_ops.rst
index fa56a0cc219..512c39b2a68 100644
--- a/advanced_source/cpp_custom_ops.rst
+++ b/advanced_source/cpp_custom_ops.rst
@@ -19,6 +19,10 @@ Custom C++ and CUDA Operators
* PyTorch 2.4 or later
* Basic understanding of C++ and CUDA programming
+.. note::
+
+ This tutorial will also work on AMD ROCm with no additional modifications.
+
PyTorch offers a large library of operators that work on Tensors (e.g. torch.add, torch.sum, etc).
However, you may wish to bring a new custom operator to PyTorch. This tutorial demonstrates the
blessed path to authoring a custom operator written in C++/CUDA.
@@ -58,14 +62,92 @@ Using ``cpp_extension`` is as simple as writing the following ``setup.py``:
setup(name="extension_cpp",
ext_modules=[
- cpp_extension.CppExtension("extension_cpp", ["muladd.cpp"])],
- cmdclass={'build_ext': cpp_extension.BuildExtension})
+ cpp_extension.CppExtension(
+ "extension_cpp",
+ ["muladd.cpp"],
+ # define Py_LIMITED_API with min version 3.9 to expose only the stable
+ # limited API subset from Python.h
+ extra_compile_args={"cxx": ["-DPy_LIMITED_API=0x03090000"]},
+ py_limited_api=True)], # Build 1 wheel across multiple Python versions
+ cmdclass={'build_ext': cpp_extension.BuildExtension},
+ options={"bdist_wheel": {"py_limited_api": "cp39"}} # 3.9 is minimum supported Python version
+ )
If you need to compile CUDA code (for example, ``.cu`` files), then instead use
`torch.utils.cpp_extension.CUDAExtension `_.
-Please see how
-`extension-cpp `_ for an example for
-how this is set up.
+Please see `extension-cpp `_ for an
+example for how this is set up.
+
+The above example represents what we refer to as a CPython agnostic wheel, meaning we are
+building a single wheel that can be run across multiple CPython versions (similar to pure
+Python packages). CPython agnosticism is desirable in minimizing the number of wheels your
+custom library needs to support and release. The minimum version we'd like to support is
+3.9, since it is the oldest supported version currently, so we use the corresponding hexcode
+and specifier throughout the setup code. We suggest building the extension in the same
+environment as the minimum CPython version you'd like to support to minimize unknown behavior,
+so, here, we build the extension in a CPython 3.9 environment. When built, this single wheel
+will be runnable in any CPython environment 3.9+. To achieve this, there are three key lines
+to note.
+
+The first is the specification of ``Py_LIMITED_API`` in ``extra_compile_args`` to the
+minimum CPython version you would like to support:
+
+.. code-block:: python
+
+ extra_compile_args={"cxx": ["-DPy_LIMITED_API=0x03090000"]},
+
+Defining the ``Py_LIMITED_API`` flag helps verify that the extension is in fact
+only using the `CPython Stable Limited API `_,
+which is a requirement for the building a CPython agnostic wheel. If this requirement
+is not met, it is possible to build a wheel that looks CPython agnostic but will crash,
+or worse, be silently incorrect, in another CPython environment. Take care to avoid
+using unstable CPython APIs, for example APIs from libtorch_python (in particular
+pytorch/python bindings,) and to only use APIs from libtorch (ATen objects, operators
+and the dispatcher). We strongly recommend defining the ``Py_LIMITED_API`` flag to
+help ascertain the extension is compliant and safe as a CPython agnostic wheel. Note that
+defining this flag is not a full guarantee that the built wheel is CPython agnostic, but
+it is better than the wild wild west. There are several caveats mentioned in the
+`Python docs `_,
+and you should test and verify yourself that the wheel is truly agnostic for the relevant
+CPython versions.
+
+The second and third lines specifying ``py_limited_api`` inform setuptools that you intend
+to build a CPython agnostic wheel and will influence the naming of the wheel accordingly:
+
+.. code-block:: python
+
+ setup(name="extension_cpp",
+ ext_modules=[
+ cpp_extension.CppExtension(
+ ...,
+ py_limited_api=True)], # Build 1 wheel across multiple Python versions
+ ...,
+ options={"bdist_wheel": {"py_limited_api": "cp39"}} # 3.9 is minimum supported Python version
+ )
+
+It is necessary to specify ``py_limited_api=True`` as an argument to CppExtension/
+CUDAExtension and also as an option to the ``"bdist_wheel"`` command with the minimal
+supported CPython version (in this case, 3.9). Consequently, the ``setup`` in our
+tutorial would build one properly named wheel that could be installed across multiple
+CPython versions ``>=3.9``.
+
+If your extension uses CPython APIs outside the stable limited set, then you cannot
+build a CPython agnostic wheel! You should build one wheel per CPython version instead,
+like so:
+
+.. code-block:: python
+
+ from setuptools import setup, Extension
+ from torch.utils import cpp_extension
+
+ setup(name="extension_cpp",
+ ext_modules=[
+ cpp_extension.CppExtension(
+ "extension_cpp",
+ ["muladd.cpp"])],
+ cmdclass={'build_ext': cpp_extension.BuildExtension},
+ )
+
Defining the custom op and adding backend implementations
---------------------------------------------------------
@@ -174,8 +256,10 @@ To add ``torch.compile`` support for an operator, we must add a FakeTensor kerne
known as a "meta kernel" or "abstract impl"). FakeTensors are Tensors that have
metadata (such as shape, dtype, device) but no data: the FakeTensor kernel for an
operator specifies how to compute the metadata of output tensors given the metadata of input tensors.
+The FakeTensor kernel should return dummy Tensors of your choice with
+the correct Tensor metadata (shape/strides/``dtype``/device).
-We recommend that this be done from Python via the `torch.library.register_fake` API,
+We recommend that this be done from Python via the ``torch.library.register_fake`` API,
though it is possible to do this from C++ as well (see
`The Custom Operators Manual `_
for more details).
@@ -186,7 +270,9 @@ for more details).
# before calling ``torch.library`` APIs that add registrations for the
# C++ custom operator(s). The following import loads our
# C++ custom operator definitions.
- # See the next section for more details.
+ # Note that if you are striving for Python agnosticism, you should use
+ # the ``load_library(...)`` API call instead. See the next section for
+ # more details.
from . import _C
@torch.library.register_fake("extension_cpp::mymuladd")
@@ -206,13 +292,89 @@ matters (importing in the wrong order will lead to an error).
To use the custom operator with hybrid Python/C++ registrations, we must
first load the C++ library that holds the custom operator definition
-and then call the ``torch.library`` registration APIs. This can happen in one
-of two ways:
+and then call the ``torch.library`` registration APIs. This can happen in
+three ways:
+
+
+1. The first way to load the C++ library that holds the custom operator definition
+ is to define a dummy Python module for _C. Then, in Python, when you import the
+ module with ``import _C``, the ``.so`` files corresponding to the extension will
+ be loaded and the ``TORCH_LIBRARY`` and ``TORCH_LIBRARY_IMPL`` static initializers
+ will run. One can create a dummy Python module with ``PYBIND11_MODULE`` like below,
+ but you will notice that this does not compile with ``Py_LIMITED_API``, because
+ ``pybind11`` does not promise to only use the stable limited CPython API! With
+ the below code, you sadly cannot build a CPython agnostic wheel for your extension!
+ (Foreshadowing: I wonder what the second way is ;) ).
+
+.. code-block:: cpp
+
+ // in, say, not_agnostic/csrc/extension_BAD.cpp
+ #include
+
+ PYBIND11_MODULE("_C", m) {}
+
+.. code-block:: python
+
+ # in, say, extension/__init__.py
+ from . import _C
+
+2. In this tutorial, because we value being able to build a single wheel across multiple
+ CPython versions, we will replace the unstable ``PYBIND11`` call with stable API calls.
+ The below code compiles with ``-DPy_LIMITED_API=0x03090000`` and successfully creates
+ a dummy Python module for our ``_C`` extension so that it can be imported from Python.
+ See `extension_cpp/__init__.py `_
+ and `extension_cpp/csrc/muladd.cpp `_
+ for more details:
+
+.. code-block:: cpp
+
+ #include
+
+ extern "C" {
+ /* Creates a dummy empty _C module that can be imported from Python.
+ The import from Python will load the .so consisting of this file
+ in this extension, so that the TORCH_LIBRARY static initializers
+ below are run. */
+ PyObject* PyInit__C(void)
+ {
+ static struct PyModuleDef module_def = {
+ PyModuleDef_HEAD_INIT,
+ "_C", /* name of module */
+ NULL, /* module documentation, may be NULL */
+ -1, /* size of per-interpreter state of the module,
+ or -1 if the module keeps state in global variables. */
+ NULL, /* methods */
+ };
+ return PyModule_Create(&module_def);
+ }
+ }
+
+.. code-block:: python
+
+ # in, say, extension/__init__.py
+ from . import _C
+
+3. If you want to avoid ``Python.h`` entirely in your C++ custom operator, you may
+ use ``torch.ops.load_library("/path/to/library.so")`` in Python to load the ``.so``
+ file(s) compiled from the extension. Note that, with this method, there is no ``_C``
+ Python module created for the extension so you cannot call ``import _C`` from Python.
+ Instead of relying on the import statement to trigger the custom operators to be
+ registered, ``torch.ops.load_library("/path/to/library.so")`` will do the trick.
+ The challenge then is shifted towards understanding where the ``.so`` files are
+ located so that you can load them, which is not always trivial:
+
+.. code-block:: python
+
+ import torch
+ from pathlib import Path
+
+ so_files = list(Path(__file__).parent.glob("_C*.so"))
+ assert (
+ len(so_files) == 1
+ ), f"Expected one _C*.so file, found {len(so_files)}"
+ torch.ops.load_library(so_files[0])
-1. If you're following this tutorial, importing the Python C extension module
- we created will load the C++ custom operator definitions.
-2. If your C++ custom operator is located in a shared library object, you can
- also use ``torch.ops.load_library("/path/to/library.so")`` to load it.
+ from . import ops
Adding training (autograd) support for an operator
@@ -417,4 +579,4 @@ Conclusion
In this tutorial, we went over the recommended approach to integrating Custom C++
and CUDA operators with PyTorch. The ``TORCH_LIBRARY/torch.library`` APIs are fairly
low-level. For more information about how to use the API, see
-`The Custom Operators Manual `_.
+`The Custom Operators Manual `_.
diff --git a/advanced_source/cpp_custom_ops_sycl.rst b/advanced_source/cpp_custom_ops_sycl.rst
new file mode 100644
index 00000000000..3b3ad069b58
--- /dev/null
+++ b/advanced_source/cpp_custom_ops_sycl.rst
@@ -0,0 +1,274 @@
+.. _cpp-custom-ops-tutorial-sycl:
+
+Custom SYCL Operators
+=====================
+
+.. grid:: 2
+
+ .. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
+ :class-card: card-prerequisites
+
+ * How to integrate custom operators written in SYCL with PyTorch
+
+ .. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
+ :class-card: card-prerequisites
+
+ * PyTorch 2.8 or later
+ * Basic understanding of SYCL programming
+
+.. note::
+
+ ``SYCL`` serves as the backend programming language for Intel GPUs (device label ``xpu``). For configuration details, see:
+ `Getting Started on Intel GPUs `_. The Intel Compiler, which comes bundled with Intel Deep Learning Essentials, handles ``SYCL`` compilation. Ensure you install and activate the compiler environment prior to executing the code examples in this tutorial.
+
+PyTorch offers a large library of operators that work on Tensors (e.g. torch.add, torch.sum, etc).
+However, you may wish to bring a new custom operator to PyTorch. This tutorial demonstrates the
+best path to authoring a custom operator written in SYCL. Tutorials for C++ and CUDA operators are available in the :ref:`cpp-custom-ops-tutorial`.
+
+Follow the structure to create a custom SYCL operator:
+
+.. code-block:: text
+
+ sycl_example/
+ ├── setup.py
+ ├── sycl_extension
+ │ ├── __init__.py
+ │ ├── muladd.sycl
+ │ └── ops.py
+ └── test_sycl_extension.py
+
+Setting up the Build System
+---------------------------
+
+If you need to compile **SYCL** code (for example, ``.sycl`` files), use `torch.utils.cpp_extension.SyclExtension `_.
+The setup process is very similar to C++/CUDA, except the compilation arguments need to be adjusted for SYCL.
+
+Using ``sycl_extension`` is as straightforward as writing the following ``setup.py``:
+
+.. code-block:: python
+
+ import os
+ import torch
+ import glob
+ from setuptools import find_packages, setup
+ from torch.utils.cpp_extension import SyclExtension, BuildExtension
+
+ library_name = "sycl_extension"
+ py_limited_api = True
+ extra_compile_args = {
+ "cxx": ["-O3",
+ "-fdiagnostics-color=always",
+ "-DPy_LIMITED_API=0x03090000"],
+ "sycl": ["-O3" ]
+ }
+
+ assert(torch.xpu.is_available()), "XPU is not available, please check your environment"
+ # Source files collection
+ this_dir = os.path.dirname(os.path.curdir)
+ extensions_dir = os.path.join(this_dir, library_name)
+ sources = list(glob.glob(os.path.join(extensions_dir, "*.sycl")))
+ # Construct extension
+ ext_modules = [
+ SyclExtension(
+ f"{library_name}._C",
+ sources,
+ extra_compile_args=extra_compile_args,
+ py_limited_api=py_limited_api,
+ )
+ ]
+ setup(
+ name=library_name,
+ packages=find_packages(),
+ ext_modules=ext_modules,
+ install_requires=["torch"],
+ description="Simple Example of PyTorch Sycl extensions",
+ cmdclass={"build_ext": BuildExtension},
+ options={"bdist_wheel": {"py_limited_api": "cp39"}} if py_limited_api else {},
+ )
+
+
+Defining the custom op and adding backend implementations
+---------------------------------------------------------
+First, let's write a SYCL function that computes ``mymuladd``:
+
+In order to use this from PyTorch’s Python frontend, we need to register it
+as a PyTorch operator using the ``TORCH_LIBRARY`` API. This will automatically
+bind the operator to Python.
+
+
+If you also have a SYCL implementation of ``myaddmul``, you can also register it
+in a separate ``TORCH_LIBRARY_IMPL`` block:
+
+.. code-block:: cpp
+
+ #include
+ #include
+ #include
+ #include
+ #include
+
+ namespace sycl_extension {
+ // MulAdd Kernel: result = a * b + c
+ static void muladd_kernel(
+ int numel, const float* a, const float* b, float c, float* result,
+ const sycl::nd_item<1>& item) {
+ int idx = item.get_global_id(0);
+ if (idx < numel) {
+ result[idx] = a[idx] * b[idx] + c;
+ }
+ }
+
+ class MulAddKernelFunctor {
+ public:
+ MulAddKernelFunctor(int _numel, const float* _a, const float* _b, float _c, float* _result)
+ : numel(_numel), a(_a), b(_b), c(_c), result(_result) {}
+ void operator()(const sycl::nd_item<1>& item) const {
+ muladd_kernel(numel, a, b, c, result, item);
+ }
+
+ private:
+ int numel;
+ const float* a;
+ const float* b;
+ float c;
+ float* result;
+ };
+
+ at::Tensor mymuladd_xpu(const at::Tensor& a, const at::Tensor& b, double c) {
+ TORCH_CHECK(a.sizes() == b.sizes(), "a and b must have the same shape");
+ TORCH_CHECK(a.dtype() == at::kFloat, "a must be a float tensor");
+ TORCH_CHECK(b.dtype() == at::kFloat, "b must be a float tensor");
+ TORCH_CHECK(a.device().is_xpu(), "a must be an XPU tensor");
+ TORCH_CHECK(b.device().is_xpu(), "b must be an XPU tensor");
+
+ at::Tensor a_contig = a.contiguous();
+ at::Tensor b_contig = b.contiguous();
+ at::Tensor result = at::empty_like(a_contig);
+
+ const float* a_ptr = a_contig.data_ptr();
+ const float* b_ptr = b_contig.data_ptr();
+ float* res_ptr = result.data_ptr();
+ int numel = a_contig.numel();
+
+ sycl::queue& queue = c10::xpu::getCurrentXPUStream().queue();
+ constexpr int threads = 256;
+ int blocks = (numel + threads - 1) / threads;
+
+ queue.submit([&](sycl::handler& cgh) {
+ cgh.parallel_for(
+ sycl::nd_range<1>(blocks * threads, threads),
+ MulAddKernelFunctor(numel, a_ptr, b_ptr, static_cast(c), res_ptr)
+ );
+ });
+
+ return result;
+ }
+ // Defines the operators
+ TORCH_LIBRARY(sycl_extension, m) {
+ m.def("mymuladd(Tensor a, Tensor b, float c) -> Tensor");
+ }
+
+ // ==================================================
+ // Register SYCL Implementations to Torch Library
+ // ==================================================
+ TORCH_LIBRARY_IMPL(sycl_extension, XPU, m) {
+ m.impl("mymuladd", &mymuladd_xpu);
+ }
+
+ } // namespace sycl_extension
+
+
+
+Create a Python Interface
+-------------------------
+
+Create a Python interface for our operator in the ``sycl_extension/ops.py`` file:
+
+.. code-block:: python
+
+ import torch
+ from torch import Tensor
+ __all__ = ["mymuladd"]
+
+ def mymuladd(a: Tensor, b: Tensor, c: float) -> Tensor:
+ """Performs a * b + c in an efficient fused kernel"""
+ return torch.ops.sycl_extension.mymuladd.default(a, b, c)
+
+Initialize Package
+------------------
+
+Create ``sycl_extension/__init__.py`` file to make the package importable:
+
+.. code-block:: python
+
+ import ctypes
+ from pathlib import Path
+
+ import torch
+
+ current_dir = Path(__file__).parent.parent
+ build_dir = current_dir / "build"
+ so_files = list(build_dir.glob("**/*.so"))
+
+ assert len(so_files) == 1, f"Expected one _C*.so file, found {len(so_files)}"
+
+ with torch._ops.dl_open_guard():
+ loaded_lib = ctypes.CDLL(so_files[0])
+
+ from . import ops
+
+ __all__ = [
+ "loaded_lib",
+ "ops",
+ ]
+
+Testing SYCL extension operator
+-------------------
+
+Use simple test to verify that the operator works correctly.
+
+.. code-block:: python
+
+ import torch
+ from torch.testing._internal.common_utils import TestCase
+ import unittest
+ import sycl_extension
+
+ def reference_muladd(a, b, c):
+ return a * b + c
+
+ class TestMyMulAdd(TestCase):
+ def sample_inputs(self, device, *, requires_grad=False):
+ def make_tensor(*size):
+ return torch.randn(size, device=device, requires_grad=requires_grad)
+
+ def make_nondiff_tensor(*size):
+ return torch.randn(size, device=device, requires_grad=False)
+
+ return [
+ [make_tensor(3), make_tensor(3), 1],
+ [make_tensor(20), make_tensor(20), 3.14],
+ [make_tensor(20), make_nondiff_tensor(20), -123],
+ [make_nondiff_tensor(2, 3), make_tensor(2, 3), -0.3],
+ ]
+
+ def _test_correctness(self, device):
+ samples = self.sample_inputs(device)
+ for args in samples:
+ result = sycl_extension.ops.mymuladd(*args)
+ expected = reference_muladd(*args)
+ torch.testing.assert_close(result, expected)
+
+ @unittest.skipIf(not torch.xpu.is_available(), "requires Intel GPU")
+ def test_correctness_xpu(self):
+ self._test_correctness("xpu")
+
+ if __name__ == "__main__":
+ unittest.main()
+
+This test checks the correctness of the custom operator by comparing its output against a reference implementation.
+
+Conclusion
+----------
+
+In this tutorial, we demonstrated how to implement and compile custom SYCL operators for PyTorch. We specifically showcased an inference operation ``muladd``. For adding backward support or enabling torch.compile compatibility, please refer to :ref:`cpp-custom-ops-tutorial`.
diff --git a/advanced_source/cpp_export.rst b/advanced_source/cpp_export.rst
index 45556a5320f..56c4bcbaae7 100644
--- a/advanced_source/cpp_export.rst
+++ b/advanced_source/cpp_export.rst
@@ -1,387 +1,3 @@
-Loading a TorchScript Model in C++
-=====================================
-
-As its name suggests, the primary interface to PyTorch is the Python
-programming language. While Python is a suitable and preferred language for
-many scenarios requiring dynamism and ease of iteration, there are equally many
-situations where precisely these properties of Python are unfavorable. One
-environment in which the latter often applies is *production* -- the land of
-low latencies and strict deployment requirements. For production scenarios, C++
-is very often the language of choice, even if only to bind it into another
-language like Java, Rust or Go. The following paragraphs will outline the path
-PyTorch provides to go from an existing Python model to a serialized
-representation that can be *loaded* and *executed* purely from C++, with no
-dependency on Python.
-
-Step 1: Converting Your PyTorch Model to Torch Script
------------------------------------------------------
-
-A PyTorch model's journey from Python to C++ is enabled by `Torch Script
-`_, a representation of a PyTorch
-model that can be understood, compiled and serialized by the Torch Script
-compiler. If you are starting out from an existing PyTorch model written in the
-vanilla "eager" API, you must first convert your model to Torch Script. In the
-most common cases, discussed below, this requires only little effort. If you
-already have a Torch Script module, you can skip to the next section of this
-tutorial.
-
-There exist two ways of converting a PyTorch model to Torch Script. The first
-is known as *tracing*, a mechanism in which the structure of the model is
-captured by evaluating it once using example inputs, and recording the flow of
-those inputs through the model. This is suitable for models that make limited
-use of control flow. The second approach is to add explicit annotations to your
-model that inform the Torch Script compiler that it may directly parse and
-compile your model code, subject to the constraints imposed by the Torch Script
-language.
-
-.. tip::
-
- You can find the complete documentation for both of these methods, as well as
- further guidance on which to use, in the official `Torch Script
- reference `_.
-
-Converting to Torch Script via Tracing
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To convert a PyTorch model to Torch Script via tracing, you must pass an
-instance of your model along with an example input to the ``torch.jit.trace``
-function. This will produce a ``torch.jit.ScriptModule`` object with the trace
-of your model evaluation embedded in the module's ``forward`` method::
-
- import torch
- import torchvision
-
- # An instance of your model.
- model = torchvision.models.resnet18()
-
- # An example input you would normally provide to your model's forward() method.
- example = torch.rand(1, 3, 224, 224)
-
- # Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
- traced_script_module = torch.jit.trace(model, example)
-
-The traced ``ScriptModule`` can now be evaluated identically to a regular
-PyTorch module::
-
- In[1]: output = traced_script_module(torch.ones(1, 3, 224, 224))
- In[2]: output[0, :5]
- Out[2]: tensor([-0.2698, -0.0381, 0.4023, -0.3010, -0.0448], grad_fn=)
-
-Converting to Torch Script via Annotation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Under certain circumstances, such as if your model employs particular forms of
-control flow, you may want to write your model in Torch Script directly and
-annotate your model accordingly. For example, say you have the following
-vanilla Pytorch model::
-
- import torch
-
- class MyModule(torch.nn.Module):
- def __init__(self, N, M):
- super(MyModule, self).__init__()
- self.weight = torch.nn.Parameter(torch.rand(N, M))
-
- def forward(self, input):
- if input.sum() > 0:
- output = self.weight.mv(input)
- else:
- output = self.weight + input
- return output
-
-
-Because the ``forward`` method of this module uses control flow that is
-dependent on the input, it is not suitable for tracing. Instead, we can convert
-it to a ``ScriptModule``.
-In order to convert the module to the ``ScriptModule``, one needs to
-compile the module with ``torch.jit.script`` as follows::
-
- class MyModule(torch.nn.Module):
- def __init__(self, N, M):
- super(MyModule, self).__init__()
- self.weight = torch.nn.Parameter(torch.rand(N, M))
-
- def forward(self, input):
- if input.sum() > 0:
- output = self.weight.mv(input)
- else:
- output = self.weight + input
- return output
-
- my_module = MyModule(10,20)
- sm = torch.jit.script(my_module)
-
-If you need to exclude some methods in your ``nn.Module``
-because they use Python features that TorchScript doesn't support yet,
-you could annotate those with ``@torch.jit.ignore``
-
-``sm`` is an instance of
-``ScriptModule`` that is ready for serialization.
-
-Step 2: Serializing Your Script Module to a File
--------------------------------------------------
-
-Once you have a ``ScriptModule`` in your hands, either from tracing or
-annotating a PyTorch model, you are ready to serialize it to a file. Later on,
-you'll be able to load the module from this file in C++ and execute it without
-any dependency on Python. Say we want to serialize the ``ResNet18`` model shown
-earlier in the tracing example. To perform this serialization, simply call
-`save `_
-on the module and pass it a filename::
-
- traced_script_module.save("traced_resnet_model.pt")
-
-This will produce a ``traced_resnet_model.pt`` file in your working directory.
-If you also would like to serialize ``sm``, call ``sm.save("my_module_model.pt")``
-We have now officially left the realm of Python and are ready to cross over to the sphere
-of C++.
-
-Step 3: Loading Your Script Module in C++
-------------------------------------------
-
-To load your serialized PyTorch model in C++, your application must depend on
-the PyTorch C++ API -- also known as *LibTorch*. The LibTorch distribution
-encompasses a collection of shared libraries, header files and CMake build
-configuration files. While CMake is not a requirement for depending on
-LibTorch, it is the recommended approach and will be well supported into the
-future. For this tutorial, we will be building a minimal C++ application using
-CMake and LibTorch that simply loads and executes a serialized PyTorch model.
-
-A Minimal C++ Application
-^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Let's begin by discussing the code to load a module. The following will already
-do:
-
-.. code-block:: cpp
-
- #include // One-stop header.
-
- #include
- #include
-
- int main(int argc, const char* argv[]) {
- if (argc != 2) {
- std::cerr << "usage: example-app \n";
- return -1;
- }
-
-
- torch::jit::script::Module module;
- try {
- // Deserialize the ScriptModule from a file using torch::jit::load().
- module = torch::jit::load(argv[1]);
- }
- catch (const c10::Error& e) {
- std::cerr << "error loading the model\n";
- return -1;
- }
-
- std::cout << "ok\n";
- }
-
-
-The ```` header encompasses all relevant includes from the
-LibTorch library necessary to run the example. Our application accepts the file
-path to a serialized PyTorch ``ScriptModule`` as its only command line argument
-and then proceeds to deserialize the module using the ``torch::jit::load()``
-function, which takes this file path as input. In return we receive a ``torch::jit::script::Module``
-object. We will examine how to execute it in a moment.
-
-Depending on LibTorch and Building the Application
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Assume we stored the above code into a file called ``example-app.cpp``. A
-minimal ``CMakeLists.txt`` to build it could look as simple as:
-
-.. code-block:: cmake
-
- cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
- project(custom_ops)
-
- find_package(Torch REQUIRED)
-
- add_executable(example-app example-app.cpp)
- target_link_libraries(example-app "${TORCH_LIBRARIES}")
- set_property(TARGET example-app PROPERTY CXX_STANDARD 17)
-
-The last thing we need to build the example application is the LibTorch
-distribution. You can always grab the latest stable release from the `download
-page /traced_resnet_model.pt
- ok
-
-Step 4: Executing the Script Module in C++
-------------------------------------------
-
-Having successfully loaded our serialized ``ResNet18`` in C++, we are now just a
-couple lines of code away from executing it! Let's add those lines to our C++
-application's ``main()`` function:
-
-.. code-block:: cpp
-
- // Create a vector of inputs.
- std::vector inputs;
- inputs.push_back(torch::ones({1, 3, 224, 224}));
-
- // Execute the model and turn its output into a tensor.
- at::Tensor output = module.forward(inputs).toTensor();
- std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';
-
-The first two lines set up the inputs to our model. We create a vector of
-``torch::jit::IValue`` (a type-erased value type ``script::Module`` methods
-accept and return) and add a single input. To create the input tensor, we use
-``torch::ones()``, the equivalent to ``torch.ones`` in the C++ API. We then
-run the ``script::Module``'s ``forward`` method, passing it the input vector we
-created. In return we get a new ``IValue``, which we convert to a tensor by
-calling ``toTensor()``.
-
-.. tip::
-
- To learn more about functions like ``torch::ones`` and the PyTorch C++ API in
- general, refer to its documentation at https://pytorch.org/cppdocs. The
- PyTorch C++ API provides near feature parity with the Python API, allowing
- you to further manipulate and process tensors just like in Python.
-
-In the last line, we print the first five entries of the output. Since we
-supplied the same input to our model in Python earlier in this tutorial, we
-should ideally see the same output. Let's try it out by re-compiling our
-application and running it with the same serialized model:
-
-.. code-block:: sh
-
- root@4b5a67132e81:/example-app/build# make
- Scanning dependencies of target example-app
- [ 50%] Building CXX object CMakeFiles/example-app.dir/example-app.cpp.o
- [100%] Linking CXX executable example-app
- [100%] Built target example-app
- root@4b5a67132e81:/example-app/build# ./example-app traced_resnet_model.pt
- -0.2698 -0.0381 0.4023 -0.3010 -0.0448
- [ Variable[CPUFloatType]{1,5} ]
-
-
-For reference, the output in Python previously was::
-
- tensor([-0.2698, -0.0381, 0.4023, -0.3010, -0.0448], grad_fn=)
-
-Looks like a good match!
-
-.. tip::
-
- To move your model to GPU memory, you can write ``model.to(at::kCUDA);``.
- Make sure the inputs to a model are also living in CUDA memory
- by calling ``tensor.to(at::kCUDA)``, which will return a new tensor in CUDA
- memory.
-
-Step 5: Getting Help and Exploring the API
-------------------------------------------
-
-This tutorial has hopefully equipped you with a general understanding of a
-PyTorch model's path from Python to C++. With the concepts described in this
-tutorial, you should be able to go from a vanilla, "eager" PyTorch model, to a
-compiled ``ScriptModule`` in Python, to a serialized file on disk and -- to
-close the loop -- to an executable ``script::Module`` in C++.
-
-Of course, there are many concepts we did not cover. For example, you may find
-yourself wanting to extend your ``ScriptModule`` with a custom operator
-implemented in C++ or CUDA, and executing this custom operator inside your
-``ScriptModule`` loaded in your pure C++ production environment. The good news
-is: this is possible, and well supported! For now, you can explore `this
-`_ folder
-for examples, and we will follow up with a tutorial shortly. In the time being,
-the following links may be generally helpful:
-
-- The Torch Script reference: https://pytorch.org/docs/master/jit.html
-- The PyTorch C++ API documentation: https://pytorch.org/cppdocs/
-- The PyTorch Python API documentation: https://pytorch.org/docs/
-
-As always, if you run into any problems or have questions, you can use our
-`forum `_ to get in touch.
+.. warning::
+ TorchScript is deprecated, please use
+ `torch.export `__ instead.
\ No newline at end of file
diff --git a/advanced_source/cpp_extension.rst b/advanced_source/cpp_extension.rst
deleted file mode 100644
index cb0e990797e..00000000000
--- a/advanced_source/cpp_extension.rst
+++ /dev/null
@@ -1,1205 +0,0 @@
-Custom C++ and CUDA Extensions
-==============================
-**Author**: `Peter Goldsborough `_. This gives you the full
-power of automatic differentiation (spares you from writing derivative
-functions) as well as the usual expressiveness of Python. However, there may be
-times when your operation is better implemented in C++. For example, your code
-may need to be *really* fast because it is called very frequently in your model
-or is very expensive even for few calls. Another plausible reason is that it
-depends on or interacts with other C or C++ libraries. To address such cases,
-PyTorch provides a very easy way of writing custom *C++ extensions*.
-
-C++ extensions are a mechanism we have developed to allow users (you) to create
-PyTorch operators defined *out-of-source*, i.e. separate from the PyTorch
-backend. This approach is *different* from the way native PyTorch operations are
-implemented. C++ extensions are intended to spare you much of the boilerplate
-associated with integrating an operation with PyTorch's backend while providing
-you with a high degree of flexibility for your PyTorch-based projects.
-Nevertheless, once you have defined your operation as a C++ extension, turning
-it into a native PyTorch function is largely a matter of code organization,
-which you can tackle after the fact if you decide to contribute your operation
-upstream.
-
-Motivation and Example
-----------------------
-
-The rest of this note will walk through a practical example of writing and using
-a C++ (and CUDA) extension. If you are being chased or someone will fire you if
-you don't get that op done by the end of the day, you can skip this section and
-head straight to the implementation details in the next section.
-
-Let's say you've come up with a new kind of recurrent unit that you found to
-have superior properties compared to the state of the art. This recurrent unit
-is similar to an LSTM, but differs in that it lacks a *forget gate* and uses an
-*Exponential Linear Unit* (ELU) as its internal activation function. Because
-this unit never forgets, we'll call it *LLTM*, or *Long-Long-Term-Memory* unit.
-
-The two ways in which LLTMs differ from vanilla LSTMs are significant enough
-that we can't configure PyTorch's ``LSTMCell`` for our purposes, so we'll have to
-create a custom cell. The first and easiest approach for this -- and likely in
-all cases a good first step -- is to implement our desired functionality in
-plain PyTorch with Python. For this, we need to subclass
-:class:`torch.nn.Module` and implement the forward pass of the LLTM. This would
-look something like this::
-
- class LLTM(torch.nn.Module):
- def __init__(self, input_features, state_size):
- super(LLTM, self).__init__()
- self.input_features = input_features
- self.state_size = state_size
- # 3 * state_size for input gate, output gate and candidate cell gate.
- # input_features + state_size because we will multiply with [input, h].
- self.weights = torch.nn.Parameter(
- torch.empty(3 * state_size, input_features + state_size))
- self.bias = torch.nn.Parameter(torch.empty(3 * state_size))
- self.reset_parameters()
-
- def reset_parameters(self):
- stdv = 1.0 / math.sqrt(self.state_size)
- for weight in self.parameters():
- weight.data.uniform_(-stdv, +stdv)
-
- def forward(self, input, state):
- old_h, old_cell = state
- X = torch.cat([old_h, input], dim=1)
-
- # Compute the input, output and candidate cell gates with one MM.
- gate_weights = F.linear(X, self.weights, self.bias)
- # Split the combined gate weight matrix into its components.
- gates = gate_weights.chunk(3, dim=1)
-
- input_gate = torch.sigmoid(gates[0])
- output_gate = torch.sigmoid(gates[1])
- # Here we use an ELU instead of the usual tanh.
- candidate_cell = F.elu(gates[2])
-
- # Compute the new cell state.
- new_cell = old_cell + candidate_cell * input_gate
- # Compute the new hidden state and output.
- new_h = torch.tanh(new_cell) * output_gate
-
- return new_h, new_cell
-
-which we could then use as expected::
-
- import torch
-
- X = torch.randn(batch_size, input_features)
- h = torch.randn(batch_size, state_size)
- C = torch.randn(batch_size, state_size)
-
- rnn = LLTM(input_features, state_size)
-
- new_h, new_C = rnn(X, (h, C))
-
-Naturally, if at all possible and plausible, you should use this approach to
-extend PyTorch. Since PyTorch has highly optimized implementations of its
-operations for CPU *and* GPU, powered by libraries such as `NVIDIA cuDNN
-`_, `Intel MKL
-`_ or `NNPACK
-`_, PyTorch code like above will often be
-fast enough. However, we can also see why, under certain circumstances, there is
-room for further performance improvements. The most obvious reason is that
-PyTorch has no knowledge of the *algorithm* you are implementing. It knows only
-of the individual operations you use to compose your algorithm. As such, PyTorch
-must execute your operations individually, one after the other. Since each
-individual call to the implementation (or *kernel*) of an operation, which may
-involve the launch of a CUDA kernel, has a certain amount of overhead, this
-overhead may become significant across many function calls. Furthermore, the
-Python interpreter that is running our code can itself slow down our program.
-
-A definite method of speeding things up is therefore to rewrite parts in C++ (or
-CUDA) and *fuse* particular groups of operations. Fusing means combining the
-implementations of many functions into a single function, which profits from
-fewer kernel launches as well as other optimizations we can perform with
-increased visibility of the global flow of data.
-
-Let's see how we can use C++ extensions to implement a *fused* version of the
-LLTM. We'll begin by writing it in plain C++, using the `ATen
-`_ library that powers much of PyTorch's
-backend, and see how easily it lets us translate our Python code. We'll then
-speed things up even more by moving parts of the model to CUDA kernel to benefit
-from the massive parallelism GPUs provide.
-
-Writing a C++ Extension
------------------------
-
-C++ extensions come in two flavors: They can be built "ahead of time" with
-:mod:`setuptools`, or "just in time" via
-:func:`torch.utils.cpp_extension.load`. We'll begin with the first approach and
-discuss the latter later.
-
-Building with :mod:`setuptools`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-For the "ahead of time" flavor, we build our C++ extension by writing a
-``setup.py`` script that uses setuptools to compile our C++ code. For the LLTM, it
-looks as simple as this::
-
- from setuptools import setup, Extension
- from torch.utils import cpp_extension
-
- setup(name='lltm_cpp',
- ext_modules=[cpp_extension.CppExtension('lltm_cpp', ['lltm.cpp'])],
- cmdclass={'build_ext': cpp_extension.BuildExtension})
-
-In this code, :class:`CppExtension` is a convenience wrapper around
-:class:`setuptools.Extension` that passes the correct include paths and sets
-the language of the extension to C++. The equivalent vanilla :mod:`setuptools`
-code would simply be::
-
- Extension(
- name='lltm_cpp',
- sources=['lltm.cpp'],
- include_dirs=cpp_extension.include_paths(),
- language='c++')
-
-:class:`BuildExtension` performs a number of required configuration steps and
-checks and also manages mixed compilation in the case of mixed C++/CUDA
-extensions. And that's all we really need to know about building C++ extensions
-for now! Let's now take a look at the implementation of our C++ extension,
-which goes into ``lltm.cpp``.
-
-Writing the C++ Op
-^^^^^^^^^^^^^^^^^^
-
-Let's start implementing the LLTM in C++! One function we'll need for the
-backward pass is the derivative of the sigmoid. This is a small enough piece of
-code to discuss the overall environment that is available to us when writing C++
-extensions:
-
-.. code-block:: cpp
-
- #include
-
- #include
-
- torch::Tensor d_sigmoid(torch::Tensor z) {
- auto s = torch::sigmoid(z);
- return (1 - s) * s;
- }
-
-```` is the one-stop header to include all the necessary PyTorch
-bits to write C++ extensions. It includes:
-
-- The ATen library, which is our primary API for tensor computation,
-- `pybind11 `_, which is how we create Python bindings for our C++ code,
-- Headers that manage the details of interaction between ATen and pybind11.
-
-The implementation of :func:`d_sigmoid` shows how to use the ATen API.
-PyTorch's tensor and variable interface is generated automatically from the
-ATen library, so we can more or less translate our Python implementation 1:1
-into C++. Our primary datatype for all computations will be
-:class:`torch::Tensor`. Its full API can be inspected `here
-`_. Notice
-also that we can include ```` or *any other C or C++ header* -- we have
-the full power of C++11 at our disposal.
-
-Note that CUDA-11.5 nvcc will hit internal compiler error while parsing torch/extension.h on Windows.
-To workaround the issue, move python binding logic to pure C++ file.
-Example use:
-
-.. code-block:: cpp
-
- #include
- at::Tensor SigmoidAlphaBlendForwardCuda(....)
-
-Instead of:
-
-.. code-block:: cpp
-
- #include
- torch::Tensor SigmoidAlphaBlendForwardCuda(...)
-
-Currently open issue for nvcc bug `here
-`_.
-Complete workaround code example `here
-`_.
-
-Forward Pass
-************
-
-Next we can port our entire forward pass to C++:
-
-.. code-block:: cpp
-
- #include
-
- std::vector lltm_forward(
- torch::Tensor input,
- torch::Tensor weights,
- torch::Tensor bias,
- torch::Tensor old_h,
- torch::Tensor old_cell) {
- auto X = torch::cat({old_h, input}, /*dim=*/1);
-
- auto gate_weights = torch::addmm(bias, X, weights.transpose(0, 1));
- auto gates = gate_weights.chunk(3, /*dim=*/1);
-
- auto input_gate = torch::sigmoid(gates[0]);
- auto output_gate = torch::sigmoid(gates[1]);
- auto candidate_cell = torch::elu(gates[2], /*alpha=*/1.0);
-
- auto new_cell = old_cell + candidate_cell * input_gate;
- auto new_h = torch::tanh(new_cell) * output_gate;
-
- return {new_h,
- new_cell,
- input_gate,
- output_gate,
- candidate_cell,
- X,
- gate_weights};
- }
-
-Backward Pass
-*************
-
-The C++ extension API currently does not provide a way of automatically
-generating a backwards function for us. As such, we have to also implement the
-backward pass of our LLTM, which computes the derivative of the loss with
-respect to each input of the forward pass. Ultimately, we will plop both the
-forward and backward function into a :class:`torch.autograd.Function` to create
-a nice Python binding. The backward function is slightly more involved, so
-we'll not dig deeper into the code (if you are interested, `Alex Graves' thesis
-`_ is a good read for more
-information on this):
-
-.. code-block:: cpp
-
- // tanh'(z) = 1 - tanh^2(z)
- torch::Tensor d_tanh(torch::Tensor z) {
- return 1 - z.tanh().pow(2);
- }
-
- // elu'(z) = relu'(z) + { alpha * exp(z) if (alpha * (exp(z) - 1)) < 0, else 0}
- torch::Tensor d_elu(torch::Tensor z, torch::Scalar alpha = 1.0) {
- auto e = z.exp();
- auto mask = (alpha * (e - 1)) < 0;
- return (z > 0).type_as(z) + mask.type_as(z) * (alpha * e);
- }
-
- std::vector lltm_backward(
- torch::Tensor grad_h,
- torch::Tensor grad_cell,
- torch::Tensor new_cell,
- torch::Tensor input_gate,
- torch::Tensor output_gate,
- torch::Tensor candidate_cell,
- torch::Tensor X,
- torch::Tensor gate_weights,
- torch::Tensor weights) {
- auto d_output_gate = torch::tanh(new_cell) * grad_h;
- auto d_tanh_new_cell = output_gate * grad_h;
- auto d_new_cell = d_tanh(new_cell) * d_tanh_new_cell + grad_cell;
-
- auto d_old_cell = d_new_cell;
- auto d_candidate_cell = input_gate * d_new_cell;
- auto d_input_gate = candidate_cell * d_new_cell;
-
- auto gates = gate_weights.chunk(3, /*dim=*/1);
- d_input_gate *= d_sigmoid(gates[0]);
- d_output_gate *= d_sigmoid(gates[1]);
- d_candidate_cell *= d_elu(gates[2]);
-
- auto d_gates =
- torch::cat({d_input_gate, d_output_gate, d_candidate_cell}, /*dim=*/1);
-
- auto d_weights = d_gates.t().mm(X);
- auto d_bias = d_gates.sum(/*dim=*/0, /*keepdim=*/true);
-
- auto d_X = d_gates.mm(weights);
- const auto state_size = grad_h.size(1);
- auto d_old_h = d_X.slice(/*dim=*/1, 0, state_size);
- auto d_input = d_X.slice(/*dim=*/1, state_size);
-
- return {d_old_h, d_input, d_weights, d_bias, d_old_cell};
- }
-
-Binding to Python
-^^^^^^^^^^^^^^^^^
-
-Once you have your operation written in C++ and ATen, you can use pybind11 to
-bind your C++ functions or classes into Python in a very simple manner.
-Questions or issues you have about this part of PyTorch C++ extensions will
-largely be addressed by `pybind11 documentation
-
-
-If we call ``help()`` on the function or module, we can see that its signature
-matches our C++ code::
-
- In[4] help(lltm_cpp.forward)
- forward(...) method of builtins.PyCapsule instance
- forward(arg0: torch::Tensor, arg1: torch::Tensor, arg2: torch::Tensor, arg3: torch::Tensor, arg4: torch::Tensor) -> List[torch::Tensor]
-
- LLTM forward
-
-Since we are now able to call our C++ functions from Python, we can wrap them
-with :class:`torch.autograd.Function` and :class:`torch.nn.Module` to make them first
-class citizens of PyTorch::
-
- import math
- import torch
-
- # Our module!
- import lltm_cpp
-
- class LLTMFunction(torch.autograd.Function):
- @staticmethod
- def forward(ctx, input, weights, bias, old_h, old_cell):
- outputs = lltm_cpp.forward(input, weights, bias, old_h, old_cell)
- new_h, new_cell = outputs[:2]
- variables = outputs[1:] + [weights]
- ctx.save_for_backward(*variables)
-
- return new_h, new_cell
-
- @staticmethod
- def backward(ctx, grad_h, grad_cell):
- outputs = lltm_cpp.backward(
- grad_h.contiguous(), grad_cell.contiguous(), *ctx.saved_tensors)
- d_old_h, d_input, d_weights, d_bias, d_old_cell = outputs
- return d_input, d_weights, d_bias, d_old_h, d_old_cell
-
-
- class LLTM(torch.nn.Module):
- def __init__(self, input_features, state_size):
- super(LLTM, self).__init__()
- self.input_features = input_features
- self.state_size = state_size
- self.weights = torch.nn.Parameter(
- torch.empty(3 * state_size, input_features + state_size))
- self.bias = torch.nn.Parameter(torch.empty(3 * state_size))
- self.reset_parameters()
-
- def reset_parameters(self):
- stdv = 1.0 / math.sqrt(self.state_size)
- for weight in self.parameters():
- weight.data.uniform_(-stdv, +stdv)
-
- def forward(self, input, state):
- return LLTMFunction.apply(input, self.weights, self.bias, *state)
-
-Performance Comparison
-**********************
-
-Now that we are able to use and call our C++ code from PyTorch, we can run a
-small benchmark to see how much performance we gained from rewriting our op in
-C++. We'll run the LLTM forwards and backwards a few times and measure the
-duration::
-
- import time
-
- import torch
-
- batch_size = 16
- input_features = 32
- state_size = 128
-
- X = torch.randn(batch_size, input_features)
- h = torch.randn(batch_size, state_size)
- C = torch.randn(batch_size, state_size)
-
- rnn = LLTM(input_features, state_size)
-
- forward = 0
- backward = 0
- for _ in range(100000):
- start = time.time()
- new_h, new_C = rnn(X, (h, C))
- forward += time.time() - start
-
- start = time.time()
- (new_h.sum() + new_C.sum()).backward()
- backward += time.time() - start
-
- print('Forward: {:.3f} s | Backward {:.3f} s'.format(forward, backward))
-
-If we run this code with the original LLTM we wrote in pure Python at the start
-of this post, we get the following numbers (on my machine)::
-
- Forward: 506.480 us | Backward 444.694 us
-
-and with our new C++ version::
-
- Forward: 349.335 us | Backward 443.523 us
-
-We can already see a significant speedup for the forward function (more than
-30%). For the backward function, a speedup is visible, albeit not a major one.
-The backward pass I wrote above was not particularly optimized and could
-definitely be improved. Also, PyTorch's automatic differentiation engine can
-automatically parallelize computation graphs, may use a more efficient flow of
-operations overall, and is also implemented in C++, so it's expected to be
-fast. Nevertheless, this is a good start.
-
-Performance on GPU Devices
-**************************
-
-A wonderful fact about PyTorch's *ATen* backend is that it abstracts the
-computing device you are running on. This means the same code we wrote for CPU
-can *also* run on GPU, and individual operations will correspondingly dispatch
-to GPU-optimized implementations. For certain operations like matrix multiply
-(like ``mm`` or ``addmm``), this is a big win. Let's take a look at how much
-performance we gain from running our C++ code with CUDA tensors. No changes to
-our implementation are required, we simply need to put our tensors in GPU
-memory from Python, with either adding ``device=cuda_device`` argument at
-creation time or using ``.to(cuda_device)`` after creation::
-
- import torch
-
- assert torch.cuda.is_available()
- cuda_device = torch.device("cuda") # device object representing GPU
-
- batch_size = 16
- input_features = 32
- state_size = 128
-
- # Note the device=cuda_device arguments here
- X = torch.randn(batch_size, input_features, device=cuda_device)
- h = torch.randn(batch_size, state_size, device=cuda_device)
- C = torch.randn(batch_size, state_size, device=cuda_device)
-
- rnn = LLTM(input_features, state_size).to(cuda_device)
-
- forward = 0
- backward = 0
- for _ in range(100000):
- start = time.time()
- new_h, new_C = rnn(X, (h, C))
- torch.cuda.synchronize()
- forward += time.time() - start
-
- start = time.time()
- (new_h.sum() + new_C.sum()).backward()
- torch.cuda.synchronize()
- backward += time.time() - start
-
- print('Forward: {:.3f} us | Backward {:.3f} us'.format(forward * 1e6/1e5, backward * 1e6/1e5))
-
-Once more comparing our plain PyTorch code with our C++ version, now both
-running on CUDA devices, we again see performance gains. For Python/PyTorch::
-
- Forward: 187.719 us | Backward 410.815 us
-
-And C++/ATen::
-
- Forward: 149.802 us | Backward 393.458 us
-
-That's a great overall speedup compared to non-CUDA code. However, we can pull
-even more performance out of our C++ code by writing custom CUDA kernels, which
-we'll dive into soon. Before that, let's discuss another way of building your C++
-extensions.
-
-JIT Compiling Extensions
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Previously, I mentioned there were two ways of building C++ extensions: using
-:mod:`setuptools` or just in time (JIT). Having covered the former, let's
-elaborate on the latter. The JIT compilation mechanism provides you with a way
-of compiling and loading your extensions on the fly by calling a simple
-function in PyTorch's API called :func:`torch.utils.cpp_extension.load`. For
-the LLTM, this would look as simple as this::
-
- from torch.utils.cpp_extension import load
-
- lltm_cpp = load(name="lltm_cpp", sources=["lltm.cpp"])
-
-Here, we provide the function with the same information as for
-:mod:`setuptools`. In the background, this will do the following:
-
-1. Create a temporary directory ``/tmp/torch_extensions/lltm``,
-2. Emit a `Ninja
-
- #include
-
- // CUDA forward declarations
-
- std::vector lltm_cuda_forward(
- torch::Tensor input,
- torch::Tensor weights,
- torch::Tensor bias,
- torch::Tensor old_h,
- torch::Tensor old_cell);
-
- std::vector lltm_cuda_backward(
- torch::Tensor grad_h,
- torch::Tensor grad_cell,
- torch::Tensor new_cell,
- torch::Tensor input_gate,
- torch::Tensor output_gate,
- torch::Tensor candidate_cell,
- torch::Tensor X,
- torch::Tensor gate_weights,
- torch::Tensor weights);
-
- // C++ interface
-
- #define CHECK_CUDA(x) TORCH_CHECK(x.device().is_cuda(), #x " must be a CUDA tensor")
- #define CHECK_CONTIGUOUS(x) TORCH_CHECK(x.is_contiguous(), #x " must be contiguous")
- #define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
-
- std::vector lltm_forward(
- torch::Tensor input,
- torch::Tensor weights,
- torch::Tensor bias,
- torch::Tensor old_h,
- torch::Tensor old_cell) {
- CHECK_INPUT(input);
- CHECK_INPUT(weights);
- CHECK_INPUT(bias);
- CHECK_INPUT(old_h);
- CHECK_INPUT(old_cell);
-
- return lltm_cuda_forward(input, weights, bias, old_h, old_cell);
- }
-
- std::vector lltm_backward(
- torch::Tensor grad_h,
- torch::Tensor grad_cell,
- torch::Tensor new_cell,
- torch::Tensor input_gate,
- torch::Tensor output_gate,
- torch::Tensor candidate_cell,
- torch::Tensor X,
- torch::Tensor gate_weights,
- torch::Tensor weights) {
- CHECK_INPUT(grad_h);
- CHECK_INPUT(grad_cell);
- CHECK_INPUT(input_gate);
- CHECK_INPUT(output_gate);
- CHECK_INPUT(candidate_cell);
- CHECK_INPUT(X);
- CHECK_INPUT(gate_weights);
- CHECK_INPUT(weights);
-
- return lltm_cuda_backward(
- grad_h,
- grad_cell,
- new_cell,
- input_gate,
- output_gate,
- candidate_cell,
- X,
- gate_weights,
- weights);
- }
-
- PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
- m.def("forward", &lltm_forward, "LLTM forward (CUDA)");
- m.def("backward", &lltm_backward, "LLTM backward (CUDA)");
- }
-
-As you can see, it is largely boilerplate, checks and forwarding to functions
-that we'll define in the CUDA file. We'll name this file
-``lltm_cuda_kernel.cu`` (note the ``.cu`` extension!). NVCC can reasonably
-compile C++11, thus we still have ATen and the C++ standard library available
-to us (but not ``torch.h``). Note that :mod:`setuptools` cannot handle files
-with the same name but different extensions, so if you use the ``setup.py``
-method instead of the JIT method, you must give your CUDA file a different name
-than your C++ file (for the JIT method, ``lltm.cpp`` and ``lltm.cu`` would work
-fine). Let's take a small peek at what this file will look like:
-
-.. code-block:: cpp
-
- #include
-
- #include
- #include
-
- #include
-
- template
- __device__ __forceinline__ scalar_t sigmoid(scalar_t z) {
- return 1.0 / (1.0 + exp(-z));
- }
-
-Here we see the headers I just described, as well as the fact that we are using
-CUDA-specific declarations like ``__device__`` and ``__forceinline__`` and
-functions like ``exp``. Let's continue with a few more helper functions that
-we'll need:
-
-.. code-block:: cpp
-
- template
- __device__ __forceinline__ scalar_t d_sigmoid(scalar_t z) {
- const auto s = sigmoid(z);
- return (1.0 - s) * s;
- }
-
- template
- __device__ __forceinline__ scalar_t d_tanh(scalar_t z) {
- const auto t = tanh(z);
- return 1 - (t * t);
- }
-
- template
- __device__ __forceinline__ scalar_t elu(scalar_t z, scalar_t alpha = 1.0) {
- return fmax(0.0, z) + fmin(0.0, alpha * (exp(z) - 1.0));
- }
-
- template
- __device__ __forceinline__ scalar_t d_elu(scalar_t z, scalar_t alpha = 1.0) {
- const auto e = exp(z);
- const auto d_relu = z < 0.0 ? 0.0 : 1.0;
- return d_relu + (((alpha * (e - 1.0)) < 0.0) ? (alpha * e) : 0.0);
- }
-
-To now actually implement a function, we'll again need two things: one function
-that performs operations we don't wish to explicitly write by hand and calls
-into CUDA kernels, and then the actual CUDA kernel for the parts we want to
-speed up. For the forward pass, the first function should look like this:
-
-.. code-block:: cpp
-
- std::vector lltm_cuda_forward(
- torch::Tensor input,
- torch::Tensor weights,
- torch::Tensor bias,
- torch::Tensor old_h,
- torch::Tensor old_cell) {
- auto X = torch::cat({old_h, input}, /*dim=*/1);
- auto gates = torch::addmm(bias, X, weights.transpose(0, 1));
-
- const auto batch_size = old_cell.size(0);
- const auto state_size = old_cell.size(1);
-
- auto new_h = torch::zeros_like(old_cell);
- auto new_cell = torch::zeros_like(old_cell);
- auto input_gate = torch::zeros_like(old_cell);
- auto output_gate = torch::zeros_like(old_cell);
- auto candidate_cell = torch::zeros_like(old_cell);
-
- const int threads = 1024;
- const dim3 blocks((state_size + threads - 1) / threads, batch_size);
-
- AT_DISPATCH_FLOATING_TYPES(gates.type(), "lltm_forward_cuda", ([&] {
- lltm_cuda_forward_kernel<<>>(
- gates.data(),
- old_cell.data(),
- new_h.data(),
- new_cell.data(),
- input_gate.data(),
- output_gate.data(),
- candidate_cell.data(),
- state_size);
- }));
-
- return {new_h, new_cell, input_gate, output_gate, candidate_cell, X, gates};
- }
-
-The main point of interest here is the ``AT_DISPATCH_FLOATING_TYPES`` macro and
-the kernel launch (indicated by the ``<<<...>>>``). While ATen abstracts away
-the device and datatype of the tensors we deal with, a tensor will, at runtime,
-still be backed by memory of a concrete type on a concrete device. As such, we
-need a way of determining at runtime what type a tensor is and then selectively
-call functions with the corresponding correct type signature. Done manually,
-this would (conceptually) look something like this:
-
-.. code-block:: cpp
-
- switch (tensor.type().scalarType()) {
- case torch::ScalarType::Double:
- return function(tensor.data());
- case torch::ScalarType::Float:
- return function(tensor.data());
- ...
- }
-
-The purpose of ``AT_DISPATCH_FLOATING_TYPES`` is to take care of this dispatch
-for us. It takes a type (``gates.type()`` in our case), a name (for error
-messages) and a lambda function. Inside this lambda function, the type alias
-``scalar_t`` is available and is defined as the type that the tensor actually
-is at runtime in that context. As such, if we have a template function (which
-our CUDA kernel will be), we can instantiate it with this ``scalar_t`` alias,
-and the correct function will be called. In this case, we also want to retrieve
-the data pointers of the tensors as pointers of that ``scalar_t`` type. If you
-wanted to dispatch over all types and not just floating point types (``Float``
-and ``Double``), you can use ``AT_DISPATCH_ALL_TYPES``.
-
-Note that we perform some operations with plain ATen. These operations will
-still run on the GPU, but using ATen's default implementations. This makes
-sense because ATen will use highly optimized routines for things like matrix
-multiplies (e.g. ``addmm``) or convolutions which would be much harder to
-implement and improve ourselves.
-
-As for the kernel launch itself, we are here specifying that each CUDA block
-will have 1024 threads, and that the entire GPU grid is split into as many
-blocks of ``1 x 1024`` threads as are required to fill our matrices with one
-thread per component. For example, if our state size was 2048 and our batch
-size 4, we'd launch a total of ``4 x 2 = 8`` blocks with each 1024 threads. If
-you've never heard of CUDA "blocks" or "grids" before, an `introductory read
-about CUDA `_ may
-help.
-
-The actual CUDA kernel is fairly simple (if you've ever programmed GPUs before):
-
-.. code-block:: cpp
-
- template
- __global__ void lltm_cuda_forward_kernel(
- const scalar_t* __restrict__ gates,
- const scalar_t* __restrict__ old_cell,
- scalar_t* __restrict__ new_h,
- scalar_t* __restrict__ new_cell,
- scalar_t* __restrict__ input_gate,
- scalar_t* __restrict__ output_gate,
- scalar_t* __restrict__ candidate_cell,
- size_t state_size) {
- const int column = blockIdx.x * blockDim.x + threadIdx.x;
- const int index = blockIdx.y * state_size + column;
- const int gates_row = blockIdx.y * (state_size * 3);
- if (column < state_size) {
- input_gate[index] = sigmoid(gates[gates_row + column]);
- output_gate[index] = sigmoid(gates[gates_row + state_size + column]);
- candidate_cell[index] = elu(gates[gates_row + 2 * state_size + column]);
- new_cell[index] =
- old_cell[index] + candidate_cell[index] * input_gate[index];
- new_h[index] = tanh(new_cell[index]) * output_gate[index];
- }
- }
-
-What's primarily interesting here is that we are able to compute all of these
-pointwise operations entirely in parallel for each individual component in our
-gate matrices. If you imagine having to do this with a giant ``for`` loop over
-a million elements in serial, you can see why this would be much faster.
-
-Using accessors
-^^^^^^^^^^^^^^^
-
-You can see in the CUDA kernel that we work directly on pointers with the right
-type. Indeed, working directly with high level type agnostic tensors inside cuda
-kernels would be very inefficient.
-
-However, this comes at a cost of ease of use and readability, especially for
-highly dimensional data. In our example, we know for example that the contiguous
-``gates`` tensor has 3 dimensions:
-
-1. batch, size of ``batch_size`` and stride of ``3*state_size``
-2. row, size of ``3`` and stride of ``state_size``
-3. index, size of ``state_size`` and stride of ``1``
-
-How can we access the element ``gates[n][row][column]`` inside the kernel then?
-It turns out that you need the strides to access your element with some simple
-arithmetic.
-
-.. code-block:: cpp
-
- gates.data()[n*3*state_size + row*state_size + column]
-
-
-In addition to being verbose, this expression needs stride to be explicitly
-known, and thus passed to the kernel function within its arguments. You can see
-that in the case of kernel functions accepting multiple tensors with different
-sizes you will end up with a very long list of arguments.
-
-Fortunately for us, ATen provides accessors that are created with a single
-dynamic check that a Tensor is the type and number of dimensions.
-Accessors then expose an API for accessing the Tensor elements efficiently
-without having to convert to a single pointer:
-
-.. code-block:: cpp
-
- torch::Tensor foo = torch::rand({12, 12});
-
- // assert foo is 2-dimensional and holds floats.
- auto foo_a = foo.accessor();
- float trace = 0;
-
- for(int i = 0; i < foo_a.size(0); i++) {
- // use the accessor foo_a to get tensor data.
- trace += foo_a[i][i];
- }
-
-Accessor objects have a relatively high level interface, with ``.size()`` and
-``.stride()`` methods and multi-dimensional indexing. The ``.accessor<>``
-interface is designed to access data efficiently on cpu tensor. The equivalent
-for cuda tensors are ``packed_accessor64<>`` and ``packed_accessor32<>``, which
-produce Packed Accessors with either 64-bit or 32-bit integer indexing.
-
-The fundamental difference with Accessor is that a Packed Accessor copies size
-and stride data inside of its structure instead of pointing to it. It allows us
-to pass it to a CUDA kernel function and use its interface inside it.
-
-We can design a function that takes Packed Accessors instead of pointers.
-
-.. code-block:: cpp
-
- __global__ void lltm_cuda_forward_kernel(
- const torch::PackedTensorAccessor32 gates,
- const torch::PackedTensorAccessor32 old_cell,
- torch::PackedTensorAccessor32 new_h,
- torch::PackedTensorAccessor32 new_cell,
- torch::PackedTensorAccessor32 input_gate,
- torch::PackedTensorAccessor32 output_gate,
- torch::PackedTensorAccessor32 candidate_cell)
-
-Let's decompose the template used here. the first two arguments ``scalar_t`` and
-``2`` are the same as regular Accessor. The argument
-``torch::RestrictPtrTraits`` indicates that the ``__restrict__`` keyword must be
-used. Note also that we've used the ``PackedAccessor32`` variant which store the
-sizes and strides in an ``int32_t``. This is important as using the 64-bit
-variant (``PackedAccessor64``) can make the kernel slower.
-
-The function declaration becomes
-
-.. code-block:: cpp
-
- template
- __global__ void lltm_cuda_forward_kernel(
- const torch::PackedTensorAccessor32 gates,
- const torch::PackedTensorAccessor32 old_cell,
- torch::PackedTensorAccessor32 new_h,
- torch::PackedTensorAccessor32 new_cell,
- torch::PackedTensorAccessor32 input_gate,
- torch::PackedTensorAccessor32 output_gate,
- torch::PackedTensorAccessor32 candidate_cell) {
- //batch index
- const int n = blockIdx.y;
- // column index
- const int c = blockIdx.x * blockDim.x + threadIdx.x;
- if (c < gates.size(2)){
- input_gate[n][c] = sigmoid(gates[n][0][c]);
- output_gate[n][c] = sigmoid(gates[n][1][c]);
- candidate_cell[n][c] = elu(gates[n][2][c]);
- new_cell[n][c] =
- old_cell[n][c] + candidate_cell[n][c] * input_gate[n][c];
- new_h[n][c] = tanh(new_cell[n][c]) * output_gate[n][c];
- }
- }
-
-The implementation is much more readable! This function is then called by
-creating Packed Accessors with the ``.packed_accessor32<>`` method within the
-host function.
-
-.. code-block:: cpp
-
- std::vector lltm_cuda_forward(
- torch::Tensor input,
- torch::Tensor weights,
- torch::Tensor bias,
- torch::Tensor old_h,
- torch::Tensor old_cell) {
- auto X = torch::cat({old_h, input}, /*dim=*/1);
- auto gate_weights = torch::addmm(bias, X, weights.transpose(0, 1));
-
- const auto batch_size = old_cell.size(0);
- const auto state_size = old_cell.size(1);
-
- auto gates = gate_weights.reshape({batch_size, 3, state_size});
- auto new_h = torch::zeros_like(old_cell);
- auto new_cell = torch::zeros_like(old_cell);
- auto input_gate = torch::zeros_like(old_cell);
- auto output_gate = torch::zeros_like(old_cell);
- auto candidate_cell = torch::zeros_like(old_cell);
-
- const int threads = 1024;
- const dim3 blocks((state_size + threads - 1) / threads, batch_size);
-
- AT_DISPATCH_FLOATING_TYPES(gates.type(), "lltm_forward_cuda", ([&] {
- lltm_cuda_forward_kernel<<>>(
- gates.packed_accessor32(),
- old_cell.packed_accessor32(),
- new_h.packed_accessor32(),
- new_cell.packed_accessor32(),
- input_gate.packed_accessor32(),
- output_gate.packed_accessor32(),
- candidate_cell.packed_accessor32());
- }));
-
- return {new_h, new_cell, input_gate, output_gate, candidate_cell, X, gates};
- }
-
-The backwards pass follows much the same pattern and I won't elaborate further
-on it:
-
-.. code-block:: cpp
-
- template
- __global__ void lltm_cuda_backward_kernel(
- torch::PackedTensorAccessor32 d_old_cell,
- torch::PackedTensorAccessor32 d_gates,
- const torch::PackedTensorAccessor32 grad_h,
- const torch::PackedTensorAccessor32 grad_cell,
- const torch::PackedTensorAccessor32 new_cell,
- const torch::PackedTensorAccessor32 input_gate,
- const torch::PackedTensorAccessor32 output_gate,
- const torch::PackedTensorAccessor32 candidate_cell,
- const torch::PackedTensorAccessor32 gate_weights) {
- //batch index
- const int n = blockIdx.y;
- // column index
- const int c = blockIdx.x * blockDim.x + threadIdx.x;
- if (c < d_gates.size(2)){
- const auto d_output_gate = tanh(new_cell[n][c]) * grad_h[n][c];
- const auto d_tanh_new_cell = output_gate[n][c] * grad_h[n][c];
- const auto d_new_cell =
- d_tanh(new_cell[n][c]) * d_tanh_new_cell + grad_cell[n][c];
-
-
- d_old_cell[n][c] = d_new_cell;
- const auto d_candidate_cell = input_gate[n][c] * d_new_cell;
- const auto d_input_gate = candidate_cell[n][c] * d_new_cell;
-
- d_gates[n][0][c] =
- d_input_gate * d_sigmoid(gate_weights[n][0][c]);
- d_gates[n][1][c] =
- d_output_gate * d_sigmoid(gate_weights[n][1][c]);
- d_gates[n][2][c] =
- d_candidate_cell * d_elu(gate_weights[n][2][c]);
- }
- }
-
- std::vector lltm_cuda_backward(
- torch::Tensor grad_h,
- torch::Tensor grad_cell,
- torch::Tensor new_cell,
- torch::Tensor input_gate,
- torch::Tensor output_gate,
- torch::Tensor candidate_cell,
- torch::Tensor X,
- torch::Tensor gates,
- torch::Tensor weights) {
- auto d_old_cell = torch::zeros_like(new_cell);
- auto d_gates = torch::zeros_like(gates);
-
- const auto batch_size = new_cell.size(0);
- const auto state_size = new_cell.size(1);
-
- const int threads = 1024;
- const dim3 blocks((state_size + threads - 1) / threads, batch_size);
-
- AT_DISPATCH_FLOATING_TYPES(X.type(), "lltm_backward_cuda", ([&] {
- lltm_cuda_backward_kernel<<>>(
- d_old_cell.packed_accessor32(),
- d_gates.packed_accessor32(),
- grad_h.packed_accessor32(),
- grad_cell.packed_accessor32(),
- new_cell.packed_accessor32(),
- input_gate.packed_accessor32(),
- output_gate.packed_accessor32(),
- candidate_cell.packed_accessor32(),
- gates.packed_accessor32());
- }));
-
- auto d_gate_weights = d_gates.reshape({batch_size, 3*state_size});
- auto d_weights = d_gate_weights.t().mm(X);
- auto d_bias = d_gate_weights.sum(/*dim=*/0, /*keepdim=*/true);
-
- auto d_X = d_gate_weights.mm(weights);
- auto d_old_h = d_X.slice(/*dim=*/1, 0, state_size);
- auto d_input = d_X.slice(/*dim=*/1, state_size);
-
- return {d_old_h, d_input, d_weights, d_bias, d_old_cell, d_gates};
- }
-
-
-Integrating a C++/CUDA Operation with PyTorch
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Integration of our CUDA-enabled op with PyTorch is again very straightforward.
-If you want to write a ``setup.py`` script, it could look like this::
-
- from setuptools import setup
- from torch.utils.cpp_extension import BuildExtension, CUDAExtension
-
- setup(
- name='lltm',
- ext_modules=[
- CUDAExtension('lltm_cuda', [
- 'lltm_cuda.cpp',
- 'lltm_cuda_kernel.cu',
- ])
- ],
- cmdclass={
- 'build_ext': BuildExtension
- })
-
-Instead of :func:`CppExtension`, we now use :func:`CUDAExtension`. We can just
-specify the ``.cu`` file along with the ``.cpp`` files -- the library takes
-care of all the hassle this entails for you. The JIT mechanism is even
-simpler::
-
- from torch.utils.cpp_extension import load
-
- lltm = load(name='lltm', sources=['lltm_cuda.cpp', 'lltm_cuda_kernel.cu'])
-
-Performance Comparison
-**********************
-
-Our hope was that parallelizing and fusing the pointwise operations of our code
-with CUDA would improve the performance of our LLTM. Let's see if that holds
-true. We can run the code I listed earlier to run a benchmark. Our fastest
-version earlier was the CUDA-based C++ code::
-
- Forward: 149.802 us | Backward 393.458 us
-
-
-And now with our custom CUDA kernel::
-
- Forward: 129.431 us | Backward 304.641 us
-
-More performance increases!
-
-Conclusion
-----------
-
-You should now be equipped with a good overview of PyTorch's C++ extension
-mechanism as well as a motivation for using them. You can find the code
-examples displayed in this note `here
-`_. If you have questions, please use
-`the forums `_. Also be sure to check our `FAQ
-`_ in case you run into any issues.
diff --git a/advanced_source/cpp_frontend.rst b/advanced_source/cpp_frontend.rst
index 901658183c7..968afa01b23 100644
--- a/advanced_source/cpp_frontend.rst
+++ b/advanced_source/cpp_frontend.rst
@@ -1,11 +1,31 @@
+.. _cpp-frontend-tutorial:
+
Using the PyTorch C++ Frontend
==============================
+**Author:** `Peter Goldsborough `_
+
+.. grid:: 2
+
+ .. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
+ :class-card: card-prerequisites
+
+ * How to build a C++ application that utilizes the PyTorch C++ frontend
+ * How to define and train neural networks from C++ using PyTorch abstractions
+
+ .. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
+ :class-card: card-prerequisites
+
+ * PyTorch 1.5 or later
+ * Basic understanding of C++ programming
+ * Basic Ubuntu Linux environment with CMake >= 3.5; similar commands will work in a MacOS / Windows environment
+ * (Optional) A CUDA-based GPU for the GPU training sections
+
The PyTorch C++ frontend is a pure C++ interface to the PyTorch machine learning
framework. While the primary interface to PyTorch naturally is Python, this
Python API sits atop a substantial C++ codebase providing foundational data
structures and functionality such as tensors and automatic differentiation. The
-C++ frontend exposes a pure C++11 API that extends this underlying C++ codebase
+C++ frontend exposes a pure C++17 API that extends this underlying C++ codebase
with tools required for machine learning training and inference. This includes a
built-in collection of common components for neural network modeling; an API to
extend this collection with custom modules; a library of popular optimization
@@ -57,7 +77,7 @@ the right tool for the job. Examples for such environments include:
Multiprocessing is an alternative, but not as scalable and has significant
shortcomings. C++ has no such constraints and threads are easy to use and
create. Models requiring heavy parallelization, like those used in `Deep
- Neuroevolution `_. We want to use a `generative adversarial
network (GAN)
`_ to solve
this task. In particular, we'll use a `DCGAN architecture
@@ -859,7 +879,7 @@ stacks them into a single tensor along the first dimension:
Note that the MNIST dataset should be located in the ``./mnist`` directory
relative to wherever you execute the training binary from. You can use `this
-script `_
+script `_
to download the MNIST dataset.
Next, we create a data loader and pass it this dataset. To make a new data
@@ -969,7 +989,7 @@ the data loader every epoch and then write the GAN training code:
discriminator->zero_grad();
torch::Tensor real_images = batch.data;
torch::Tensor real_labels = torch::empty(batch.data.size(0)).uniform_(0.8, 1.0);
- torch::Tensor real_output = discriminator->forward(real_images);
+ torch::Tensor real_output = discriminator->forward(real_images).reshape(real_labels.sizes());
torch::Tensor d_loss_real = torch::binary_cross_entropy(real_output, real_labels);
d_loss_real.backward();
@@ -977,7 +997,7 @@ the data loader every epoch and then write the GAN training code:
torch::Tensor noise = torch::randn({batch.data.size(0), kNoiseSize, 1, 1});
torch::Tensor fake_images = generator->forward(noise);
torch::Tensor fake_labels = torch::zeros(batch.data.size(0));
- torch::Tensor fake_output = discriminator->forward(fake_images.detach());
+ torch::Tensor fake_output = discriminator->forward(fake_images.detach()).reshape(fake_labels.sizes());
torch::Tensor d_loss_fake = torch::binary_cross_entropy(fake_output, fake_labels);
d_loss_fake.backward();
@@ -987,7 +1007,7 @@ the data loader every epoch and then write the GAN training code:
// Train generator.
generator->zero_grad();
fake_labels.fill_(1);
- fake_output = discriminator->forward(fake_images);
+ fake_output = discriminator->forward(fake_images).reshape(fake_labels.sizes());
torch::Tensor g_loss = torch::binary_cross_entropy(fake_output, fake_labels);
g_loss.backward();
generator_optimizer.step();
diff --git a/advanced_source/custom_class_pt2.rst b/advanced_source/custom_class_pt2.rst
new file mode 100644
index 00000000000..229a94f2ce9
--- /dev/null
+++ b/advanced_source/custom_class_pt2.rst
@@ -0,0 +1,275 @@
+Supporting Custom C++ Classes in torch.compile/torch.export
+===========================================================
+
+
+This tutorial is a follow-on to the
+:doc:`custom C++ classes ` tutorial, and
+introduces additional steps that are needed to support custom C++ classes in
+torch.compile/torch.export.
+
+.. warning::
+
+ This feature is in prototype status and is subject to backwards compatibility
+ breaking changes. This tutorial provides a snapshot as of PyTorch 2.8. If
+ you run into any issues, please reach out to us on Github!
+
+Concretely, there are a few steps:
+
+1. Implement an ``__obj_flatten__`` method to the C++ custom class
+ implementation to allow us to inspect its states and guard the changes. The
+ method should return a tuple of tuple of attribute_name, value
+ (``tuple[tuple[str, value] * n]``).
+
+2. Register a python fake class using ``@torch._library.register_fake_class``
+
+ a. Implement “fake methods” of each of the class’s c++ methods, which should
+ have the same schema as the C++ implementation.
+
+ b. Additionally, implement an ``__obj_unflatten__`` classmethod in the Python
+ fake class to tell us how to create a fake class from the flattened
+ states returned by ``__obj_flatten__``.
+
+Here is a breakdown of the diff. Following the guide in
+:doc:`Extending TorchScript with Custom C++ Classes `,
+we can create a thread-safe tensor queue and build it.
+
+.. code-block:: cpp
+
+ // Thread-safe Tensor Queue
+
+ #include
+ #include
+
+ #include
+ #include
+ #include
+
+ using namespace torch::jit;
+
+ // Thread-safe Tensor Queue
+ struct TensorQueue : torch::CustomClassHolder {
+ explicit TensorQueue(at::Tensor t) : init_tensor_(t) {}
+
+ explicit TensorQueue(c10::Dict dict) {
+ init_tensor_ = dict.at(std::string("init_tensor"));
+ const std::string key = "queue";
+ at::Tensor size_tensor;
+ size_tensor = dict.at(std::string(key + "/size")).cpu();
+ const auto* size_tensor_acc = size_tensor.const_data_ptr();
+ int64_t queue_size = size_tensor_acc[0];
+
+ for (const auto index : c10::irange(queue_size)) {
+ at::Tensor val;
+ queue_[index] = dict.at(key + "/" + std::to_string(index));
+ queue_.push_back(val);
+ }
+ }
+
+ // Push the element to the rear of queue.
+ // Lock is added for thread safe.
+ void push(at::Tensor x) {
+ std::lock_guard guard(mutex_);
+ queue_.push_back(x);
+ }
+ // Pop the front element of queue and return it.
+ // If empty, return init_tensor_.
+ // Lock is added for thread safe.
+ at::Tensor pop() {
+ std::lock_guard guard(mutex_);
+ if (!queue_.empty()) {
+ auto val = queue_.front();
+ queue_.pop_front();
+ return val;
+ } else {
+ return init_tensor_;
+ }
+ }
+
+ std::vector get_raw_queue() {
+ std::vector raw_queue(queue_.begin(), queue_.end());
+ return raw_queue;
+ }
+
+ private:
+ std::deque queue_;
+ std::mutex mutex_;
+ at::Tensor init_tensor_;
+ };
+
+ // The torch binding code
+ TORCH_LIBRARY(MyCustomClass, m) {
+ m.class_("TensorQueue")
+ .def(torch::init())
+ .def("push", &TensorQueue::push)
+ .def("pop", &TensorQueue::pop)
+ .def("get_raw_queue", &TensorQueue::get_raw_queue);
+ }
+
+**Step 1**: Add an ``__obj_flatten__`` method to the C++ custom class implementation:
+
+.. code-block:: cpp
+
+ // Thread-safe Tensor Queue
+ struct TensorQueue : torch::CustomClassHolder {
+ ...
+ std::tuple>, std::tuple> __obj_flatten__() {
+ return std::tuple(std::tuple("queue", this->get_raw_queue()), std::tuple("init_tensor_", this->init_tensor_.clone()));
+ }
+ ...
+ };
+
+ TORCH_LIBRARY(MyCustomClass, m) {
+ m.class_("TensorQueue")
+ .def(torch::init())
+ ...
+ .def("__obj_flatten__", &TensorQueue::__obj_flatten__);
+ }
+
+**Step 2a**: Register a fake class in Python that implements each method.
+
+.. code-block:: python
+
+ # namespace::class_name
+ @torch._library.register_fake_class("MyCustomClass::TensorQueue")
+ class FakeTensorQueue:
+ def __init__(
+ self,
+ queue: List[torch.Tensor],
+ init_tensor_: torch.Tensor
+ ) -> None:
+ self.queue = queue
+ self.init_tensor_ = init_tensor_
+
+ def push(self, tensor: torch.Tensor) -> None:
+ self.queue.append(tensor)
+
+ def pop(self) -> torch.Tensor:
+ if len(self.queue) > 0:
+ return self.queue.pop(0)
+ return self.init_tensor_
+
+**Step 2b**: Implement an ``__obj_unflatten__`` classmethod in Python.
+
+.. code-block:: python
+
+ # namespace::class_name
+ @torch._library.register_fake_class("MyCustomClass::TensorQueue")
+ class FakeTensorQueue:
+ ...
+ @classmethod
+ def __obj_unflatten__(cls, flattened_tq):
+ return cls(**dict(flattened_tq))
+
+
+That’s it! Now we can create a module that uses this object and run it with ``torch.compile`` or ``torch.export``.
+
+.. code-block:: python
+
+ import torch
+
+ torch.classes.load_library("build/libcustom_class.so")
+ tq = torch.classes.MyCustomClass.TensorQueue(torch.empty(0).fill_(-1))
+
+ class Mod(torch.nn.Module):
+ def forward(self, tq, x):
+ tq.push(x.sin())
+ tq.push(x.cos())
+ poped_t = tq.pop()
+ assert torch.allclose(poped_t, x.sin())
+ return tq, poped_t
+
+ tq, poped_t = torch.compile(Mod(), backend="eager", fullgraph=True)(tq, torch.randn(2, 3))
+ assert tq.size() == 1
+
+ exported_program = torch.export.export(Mod(), (tq, torch.randn(2, 3),), strict=False)
+ exported_program.module()(tq, torch.randn(2, 3))
+
+We can also implement custom ops that take custom classes as inputs. For
+example, we could register a custom op ``for_each_add_(tq, tensor)``
+
+.. code-block:: cpp
+
+ struct TensorQueue : torch::CustomClassHolder {
+ ...
+ void for_each_add_(at::Tensor inc) {
+ for (auto& t : queue_) {
+ t.add_(inc);
+ }
+ }
+ ...
+ }
+
+
+ TORCH_LIBRARY_FRAGMENT(MyCustomClass, m) {
+ m.class_("TensorQueue")
+ ...
+ .def("for_each_add_", &TensorQueue::for_each_add_);
+
+ m.def(
+ "for_each_add_(__torch__.torch.classes.MyCustomClass.TensorQueue foo, Tensor inc) -> ()");
+ }
+
+ void for_each_add_(c10::intrusive_ptr tq, at::Tensor inc) {
+ tq->for_each_add_(inc);
+ }
+
+ TORCH_LIBRARY_IMPL(MyCustomClass, CPU, m) {
+ m.impl("for_each_add_", for_each_add_);
+ }
+
+
+Since the fake class is implemented in python, we require the fake
+implementation of custom op must also be registered in python:
+
+.. code-block:: python
+
+ @torch.library.register_fake("MyCustomClass::for_each_add_")
+ def fake_for_each_add_(tq, inc):
+ tq.for_each_add_(inc)
+
+After re-compilation, we can export the custom op with:
+
+.. code-block:: python
+
+ class ForEachAdd(torch.nn.Module):
+ def forward(self, tq: torch.ScriptObject, a: torch.Tensor) -> torch.ScriptObject:
+ torch.ops.MyCustomClass.for_each_add_(tq, a)
+ return tq
+
+ mod = ForEachAdd()
+ tq = empty_tensor_queue()
+ qlen = 10
+ for i in range(qlen):
+ tq.push(torch.zeros(1))
+
+ ep = torch.export.export(mod, (tq, torch.ones(1)), strict=False)
+
+Why do we need to make a Fake Class?
+------------------------------------
+
+Tracing with real custom object has several major downsides:
+
+1. Operators on real objects can be time consuming e.g. the custom object
+ might be reading from the network or loading data from the disk.
+
+2. We don’t want to mutate the real custom object or create side-effects to the environment while tracing.
+
+3. It cannot support dynamic shapes.
+
+However, it may be difficult for users to write a fake class, e.g. if the
+original class uses some third-party library that determines the output shape of
+the methods, or is complicated and written by others. In such cases, users can
+disable the fakification requirement by defining a ``tracing_mode`` method to
+return ``"real"``:
+
+.. code-block:: cpp
+
+ std::string tracing_mode() {
+ return "real";
+ }
+
+
+A caveat of fakification is regarding **tensor aliasing.** We assume that no
+tensors within a torchbind object aliases a tensor outside of the torchbind
+object. Therefore, mutating one of these tensors will result in undefined
+behavior.
diff --git a/advanced_source/torch_script_custom_classes.rst b/advanced_source/custom_classes.rst
similarity index 51%
rename from advanced_source/torch_script_custom_classes.rst
rename to advanced_source/custom_classes.rst
index cccb86ff4ce..014bac2eebf 100644
--- a/advanced_source/torch_script_custom_classes.rst
+++ b/advanced_source/custom_classes.rst
@@ -1,10 +1,9 @@
-Extending TorchScript with Custom C++ Classes
+Extending PyTorch with Custom C++ Classes
===============================================
-This tutorial is a follow-on to the
-:doc:`custom operator `
-tutorial, and introduces the API we've built for binding C++ classes into TorchScript
-and Python simultaneously. The API is very similar to
+
+This tutorial introduces an API for binding C++ classes into PyTorch.
+The API is very similar to
`pybind11 `_, and most of the concepts will transfer
over if you're familiar with that system.
@@ -14,14 +13,14 @@ Implementing and Binding the Class in C++
For this tutorial, we are going to define a simple C++ class that maintains persistent
state in a member variable.
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/class.cpp
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/class.cpp
:language: cpp
:start-after: BEGIN class
:end-before: END class
There are several things to note:
-- ``torch/custom_class.h`` is the header you need to include to extend TorchScript
+- ``torch/custom_class.h`` is the header you need to include to extend PyTorch
with your custom class.
- Notice that whenever we are working with instances of the custom
class, we do it via instances of ``c10::intrusive_ptr<>``. Think of ``intrusive_ptr``
@@ -34,10 +33,10 @@ There are several things to note:
``torch::CustomClassHolder``. This ensures that the custom class has space to
store the reference count.
-Now let's take a look at how we will make this class visible to TorchScript, a process called
+Now let's take a look at how we will make this class visible to PyTorch, a process called
*binding* the class:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/class.cpp
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/class.cpp
:language: cpp
:start-after: BEGIN binding
:end-before: END binding
@@ -56,7 +55,7 @@ we've covered so far and place it in a file called ``class.cpp``.
Then, write a simple ``CMakeLists.txt`` file and place it in the
same directory. Here is what ``CMakeLists.txt`` should look like:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/CMakeLists.txt
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/CMakeLists.txt
:language: cmake
Also, create a ``build`` directory. Your file tree should look like this::
@@ -66,8 +65,6 @@ Also, create a ``build`` directory. Your file tree should look like this::
CMakeLists.txt
build/
-We assume you've setup your environment in the same way as described in
-the :doc:`previous tutorial `.
Go ahead and invoke cmake and then make to build the project:
.. code-block:: shell
@@ -117,137 +114,16 @@ file present in the build directory. On Linux, this is probably named
build/
libcustom_class.so
-Using the C++ Class from Python and TorchScript
+Using the C++ Class from Python
-----------------------------------------------
Now that we have our class and its registration compiled into an ``.so`` file,
we can load that `.so` into Python and try it out. Here's a script that
demonstrates that:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/custom_test.py
- :language: python
-
-
-Saving, Loading, and Running TorchScript Code Using Custom Classes
-------------------------------------------------------------------
-
-We can also use custom-registered C++ classes in a C++ process using
-libtorch. As an example, let's define a simple ``nn.Module`` that
-instantiates and calls a method on our MyStackClass class:
-
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/save.py
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/custom_test.py
:language: python
-``foo.pt`` in our filesystem now contains the serialized TorchScript
-program we've just defined.
-
-Now, we're going to define a new CMake project to show how you can load
-this model and its required .so file. For a full treatment of how to do this,
-please have a look at the `Loading a TorchScript Model in C++ Tutorial `_.
-
-Similarly to before, let's create a file structure containing the following::
-
- cpp_inference_example/
- infer.cpp
- CMakeLists.txt
- foo.pt
- build/
- custom_class_project/
- class.cpp
- CMakeLists.txt
- build/
-
-Notice we've copied over the serialized ``foo.pt`` file, as well as the source
-tree from the ``custom_class_project`` above. We will be adding the
-``custom_class_project`` as a dependency to this C++ project so that we can
-build the custom class into the binary.
-
-Let's populate ``infer.cpp`` with the following:
-
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/infer.cpp
- :language: cpp
-
-And similarly let's define our CMakeLists.txt file:
-
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/CMakeLists.txt
- :language: cpp
-
-You know the drill: ``cd build``, ``cmake``, and ``make``:
-
-.. code-block:: shell
-
- $ cd build
- $ cmake -DCMAKE_PREFIX_PATH="$(python -c 'import torch.utils; print(torch.utils.cmake_prefix_path)')" ..
- -- The C compiler identification is GNU 7.3.1
- -- The CXX compiler identification is GNU 7.3.1
- -- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc
- -- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc -- works
- -- Detecting C compiler ABI info
- -- Detecting C compiler ABI info - done
- -- Detecting C compile features
- -- Detecting C compile features - done
- -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++
- -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++ -- works
- -- Detecting CXX compiler ABI info
- -- Detecting CXX compiler ABI info - done
- -- Detecting CXX compile features
- -- Detecting CXX compile features - done
- -- Looking for pthread.h
- -- Looking for pthread.h - found
- -- Looking for pthread_create
- -- Looking for pthread_create - not found
- -- Looking for pthread_create in pthreads
- -- Looking for pthread_create in pthreads - not found
- -- Looking for pthread_create in pthread
- -- Looking for pthread_create in pthread - found
- -- Found Threads: TRUE
- -- Found torch: /local/miniconda3/lib/python3.7/site-packages/torch/lib/libtorch.so
- -- Configuring done
- -- Generating done
- -- Build files have been written to: /cpp_inference_example/build
- $ make -j
- Scanning dependencies of target custom_class
- [ 25%] Building CXX object custom_class_project/CMakeFiles/custom_class.dir/class.cpp.o
- [ 50%] Linking CXX shared library libcustom_class.so
- [ 50%] Built target custom_class
- Scanning dependencies of target infer
- [ 75%] Building CXX object CMakeFiles/infer.dir/infer.cpp.o
- [100%] Linking CXX executable infer
- [100%] Built target infer
-
-And now we can run our exciting C++ binary:
-
-.. code-block:: shell
-
- $ ./infer
- momfoobarbaz
-
-Incredible!
-
-Moving Custom Classes To/From IValues
--------------------------------------
-
-It's also possible that you may need to move custom classes into or out of
-``IValue``s, such as when you take or return ``IValue``s from TorchScript methods
-or you want to instantiate a custom class attribute in C++. For creating an
-``IValue`` from a custom C++ class instance:
-
-- ``torch::make_custom_class()`` provides an API similar to c10::intrusive_ptr
- in that it will take whatever set of arguments you provide to it, call the constructor
- of T that matches that set of arguments, and wrap that instance up and return it.
- However, instead of returning just a pointer to a custom class object, it returns
- an ``IValue`` wrapping the object. You can then pass this ``IValue`` directly to
- TorchScript.
-- In the event that you already have an ``intrusive_ptr`` pointing to your class, you
- can directly construct an IValue from it using the constructor ``IValue(intrusive_ptr)``.
-
-For converting ``IValue`` back to custom classes:
-
-- ``IValue::toCustomClass()`` will return an ``intrusive_ptr`` pointing to the
- custom class that the ``IValue`` contains. Internally, this function is checking
- that ``T`` is registered as a custom class and that the ``IValue`` does in fact contain
- a custom class. You can check whether the ``IValue`` contains a custom class manually by
- calling ``isCustomClass()``.
Defining Serialization/Deserialization Methods for Custom C++ Classes
---------------------------------------------------------------------
@@ -255,7 +131,7 @@ Defining Serialization/Deserialization Methods for Custom C++ Classes
If you try to save a ``ScriptModule`` with a custom-bound C++ class as
an attribute, you'll get the following error:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/export_attr.py
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/export_attr.py
:language: python
.. code-block:: shell
@@ -263,13 +139,13 @@ an attribute, you'll get the following error:
$ python export_attr.py
RuntimeError: Cannot serialize custom bound C++ class __torch__.torch.classes.my_classes.MyStackClass. Please define serialization methods via def_pickle for this class. (pushIValueImpl at ../torch/csrc/jit/pickler.cpp:128)
-This is because TorchScript cannot automatically figure out what information
+This is because PyTorch cannot automatically figure out what information
save from your C++ class. You must specify that manually. The way to do that
is to define ``__getstate__`` and ``__setstate__`` methods on the class using
the special ``def_pickle`` method on ``class_``.
.. note::
- The semantics of ``__getstate__`` and ``__setstate__`` in TorchScript are
+ The semantics of ``__getstate__`` and ``__setstate__`` are
equivalent to that of the Python pickle module. You can
`read more `_
about how we use these methods.
@@ -277,7 +153,7 @@ the special ``def_pickle`` method on ``class_``.
Here is an example of the ``def_pickle`` call we can add to the registration of
``MyStackClass`` to include serialization methods:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/class.cpp
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/class.cpp
:language: cpp
:start-after: BEGIN def_pickle
:end-before: END def_pickle
@@ -303,7 +179,7 @@ Once you've defined a custom C++ class, you can also use that class
as an argument or return from a custom operator (i.e. free functions). Suppose
you have the following free function:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/class.cpp
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/class.cpp
:language: cpp
:start-after: BEGIN free_function
:end-before: END free_function
@@ -311,14 +187,11 @@ you have the following free function:
You can register it running the following code inside your ``TORCH_LIBRARY``
block:
-.. literalinclude:: ../advanced_source/torch_script_custom_classes/custom_class_project/class.cpp
+.. literalinclude:: ../advanced_source/custom_classes/custom_class_project/class.cpp
:language: cpp
:start-after: BEGIN def_free
:end-before: END def_free
-Refer to the `custom op tutorial `_
-for more details on the registration API.
-
Once this is done, you can use the op like the following example:
.. code-block:: python
@@ -344,13 +217,12 @@ Once this is done, you can use the op like the following example:
Conclusion
----------
-This tutorial walked you through how to expose a C++ class to TorchScript
-(and by extension Python), how to register its methods, how to use that
-class from Python and TorchScript, and how to save and load code using
-the class and run that code in a standalone C++ process. You are now ready
-to extend your TorchScript models with C++ classes that interface with
-third party C++ libraries or implement any other use case that requires the
-lines between Python, TorchScript and C++ to blend smoothly.
+This tutorial walked you through how to expose a C++ class to PyTorch, how to
+register its methods, how to use that class from Python, and how to save and
+load code using the class and run that code in a standalone C++ process. You
+are now ready to extend your PyTorch models with C++ classes that interface
+with third party C++ libraries or implement any other use case that requires
+the lines between Python and C++ to blend smoothly.
As always, if you run into any problems or have questions, you can use our
`forum `_ or C++ ``TORCH_LIBRARY``
APIs.
-TL;DR
------
+
Authoring a custom operator from Python
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -19,17 +18,24 @@ Authoring a custom operator from Python
Please see :ref:`python-custom-ops-tutorial`.
You may wish to author a custom operator from Python (as opposed to C++) if:
+
- you have a Python function you want PyTorch to treat as an opaque callable, especially with
-respect to ``torch.compile`` and ``torch.export``.
+ respect to ``torch.compile`` and ``torch.export``.
- you have some Python bindings to C++/CUDA kernels and want those to compose with PyTorch
-subsystems (like ``torch.compile`` or ``torch.autograd``)
+ subsystems (like ``torch.compile`` or ``torch.autograd``)
+- you are using Python (and not a C++-only environment like AOTInductor).
Integrating custom C++ and/or CUDA code with PyTorch
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Please see :ref:`cpp-custom-ops-tutorial`.
+.. note::
+
+ ``SYCL`` serves as the backend programming language for Intel GPUs. Integrate custom Sycl code refer to :ref:`cpp-custom-ops-tutorial-sycl`.
+
You may wish to author a custom operator from C++ (as opposed to Python) if:
+
- you have custom C++ and/or CUDA code.
- you plan to use this code with ``AOTInductor`` to do Python-less inference.
diff --git a/advanced_source/ddp_pipeline.rst b/advanced_source/ddp_pipeline.rst
new file mode 100644
index 00000000000..bf9e4d28f33
--- /dev/null
+++ b/advanced_source/ddp_pipeline.rst
@@ -0,0 +1,10 @@
+Training Transformer models using Distributed Data Parallel and Pipeline Parallelism
+====================================================================================
+
+This tutorial has been deprecated.
+
+Redirecting to the latest parallelism APIs in 3 seconds...
+
+.. raw:: html
+
+ `_
-
-**Edited by**: `Seth Weidman `_ -
-to an LSTM-based next word-prediction model, closely following the
-`word language model `_
-from the PyTorch examples.
-"""
-
-# imports
-import os
-from io import open
-import time
-
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-
-######################################################################
-# 1. Define the model
-# -------------------
-#
-# Here we define the LSTM model architecture, following the
-# `model `_
-# from the word language model example.
-
-class LSTMModel(nn.Module):
- """Container module with an encoder, a recurrent module, and a decoder."""
-
- def __init__(self, ntoken, ninp, nhid, nlayers, dropout=0.5):
- super(LSTMModel, self).__init__()
- self.drop = nn.Dropout(dropout)
- self.encoder = nn.Embedding(ntoken, ninp)
- self.rnn = nn.LSTM(ninp, nhid, nlayers, dropout=dropout)
- self.decoder = nn.Linear(nhid, ntoken)
-
- self.init_weights()
-
- self.nhid = nhid
- self.nlayers = nlayers
-
- def init_weights(self):
- initrange = 0.1
- self.encoder.weight.data.uniform_(-initrange, initrange)
- self.decoder.bias.data.zero_()
- self.decoder.weight.data.uniform_(-initrange, initrange)
-
- def forward(self, input, hidden):
- emb = self.drop(self.encoder(input))
- output, hidden = self.rnn(emb, hidden)
- output = self.drop(output)
- decoded = self.decoder(output)
- return decoded, hidden
-
- def init_hidden(self, bsz):
- weight = next(self.parameters())
- return (weight.new_zeros(self.nlayers, bsz, self.nhid),
- weight.new_zeros(self.nlayers, bsz, self.nhid))
-
-######################################################################
-# 2. Load in the text data
-# ------------------------
-#
-# Next, we load the
-# `Wikitext-2 dataset `_ into a `Corpus`,
-# again following the
-# `preprocessing `_
-# from the word language model example.
-
-class Dictionary(object):
- def __init__(self):
- self.word2idx = {}
- self.idx2word = []
-
- def add_word(self, word):
- if word not in self.word2idx:
- self.idx2word.append(word)
- self.word2idx[word] = len(self.idx2word) - 1
- return self.word2idx[word]
-
- def __len__(self):
- return len(self.idx2word)
-
-
-class Corpus(object):
- def __init__(self, path):
- self.dictionary = Dictionary()
- self.train = self.tokenize(os.path.join(path, 'train.txt'))
- self.valid = self.tokenize(os.path.join(path, 'valid.txt'))
- self.test = self.tokenize(os.path.join(path, 'test.txt'))
-
- def tokenize(self, path):
- """Tokenizes a text file."""
- assert os.path.exists(path)
- # Add words to the dictionary
- with open(path, 'r', encoding="utf8") as f:
- for line in f:
- words = line.split() + ['']
- for word in words:
- self.dictionary.add_word(word)
-
- # Tokenize file content
- with open(path, 'r', encoding="utf8") as f:
- idss = []
- for line in f:
- words = line.split() + ['']
- ids = []
- for word in words:
- ids.append(self.dictionary.word2idx[word])
- idss.append(torch.tensor(ids).type(torch.int64))
- ids = torch.cat(idss)
-
- return ids
-
-model_data_filepath = 'data/'
-
-corpus = Corpus(model_data_filepath + 'wikitext-2')
-
-######################################################################
-# 3. Load the pretrained model
-# -----------------------------
-#
-# This is a tutorial on dynamic quantization, a quantization technique
-# that is applied after a model has been trained. Therefore, we'll simply load some
-# pretrained weights into this model architecture; these weights were obtained
-# by training for five epochs using the default settings in the word language model
-# example.
-
-ntokens = len(corpus.dictionary)
-
-model = LSTMModel(
- ntoken = ntokens,
- ninp = 512,
- nhid = 256,
- nlayers = 5,
-)
-
-model.load_state_dict(
- torch.load(
- model_data_filepath + 'word_language_model_quantize.pth',
- map_location=torch.device('cpu')
- )
- )
-
-model.eval()
-print(model)
-
-######################################################################
-# Now let's generate some text to ensure that the pretrained model is working
-# properly - similarly to before, we follow
-# `here `_
-
-input_ = torch.randint(ntokens, (1, 1), dtype=torch.long)
-hidden = model.init_hidden(1)
-temperature = 1.0
-num_words = 1000
-
-with open(model_data_filepath + 'out.txt', 'w') as outf:
- with torch.no_grad(): # no tracking history
- for i in range(num_words):
- output, hidden = model(input_, hidden)
- word_weights = output.squeeze().div(temperature).exp().cpu()
- word_idx = torch.multinomial(word_weights, 1)[0]
- input_.fill_(word_idx)
-
- word = corpus.dictionary.idx2word[word_idx]
-
- outf.write(str(word.encode('utf-8')) + ('\n' if i % 20 == 19 else ' '))
-
- if i % 100 == 0:
- print('| Generated {}/{} words'.format(i, 1000))
-
-with open(model_data_filepath + 'out.txt', 'r') as outf:
- all_output = outf.read()
- print(all_output)
-
-######################################################################
-# It's no GPT-2, but it looks like the model has started to learn the structure of
-# language!
-#
-# We're almost ready to demonstrate dynamic quantization. We just need to define a few more
-# helper functions:
-
-bptt = 25
-criterion = nn.CrossEntropyLoss()
-eval_batch_size = 1
-
-# create test data set
-def batchify(data, bsz):
- # Work out how cleanly we can divide the dataset into ``bsz`` parts.
- nbatch = data.size(0) // bsz
- # Trim off any extra elements that wouldn't cleanly fit (remainders).
- data = data.narrow(0, 0, nbatch * bsz)
- # Evenly divide the data across the ``bsz`` batches.
- return data.view(bsz, -1).t().contiguous()
-
-test_data = batchify(corpus.test, eval_batch_size)
-
-# Evaluation functions
-def get_batch(source, i):
- seq_len = min(bptt, len(source) - 1 - i)
- data = source[i:i+seq_len]
- target = source[i+1:i+1+seq_len].reshape(-1)
- return data, target
-
-def repackage_hidden(h):
- """Wraps hidden states in new Tensors, to detach them from their history."""
-
- if isinstance(h, torch.Tensor):
- return h.detach()
- else:
- return tuple(repackage_hidden(v) for v in h)
-
-def evaluate(model_, data_source):
- # Turn on evaluation mode which disables dropout.
- model_.eval()
- total_loss = 0.
- hidden = model_.init_hidden(eval_batch_size)
- with torch.no_grad():
- for i in range(0, data_source.size(0) - 1, bptt):
- data, targets = get_batch(data_source, i)
- output, hidden = model_(data, hidden)
- hidden = repackage_hidden(hidden)
- output_flat = output.view(-1, ntokens)
- total_loss += len(data) * criterion(output_flat, targets).item()
- return total_loss / (len(data_source) - 1)
-
-######################################################################
-# 4. Test dynamic quantization
-# ----------------------------
-#
-# Finally, we can call ``torch.quantization.quantize_dynamic`` on the model!
-# Specifically,
-#
-# - We specify that we want the ``nn.LSTM`` and ``nn.Linear`` modules in our
-# model to be quantized
-# - We specify that we want weights to be converted to ``int8`` values
-
-import torch.quantization
-
-quantized_model = torch.quantization.quantize_dynamic(
- model, {nn.LSTM, nn.Linear}, dtype=torch.qint8
-)
-print(quantized_model)
-
-######################################################################
-# The model looks the same; how has this benefited us? First, we see a
-# significant reduction in model size:
-
-def print_size_of_model(model):
- torch.save(model.state_dict(), "temp.p")
- print('Size (MB):', os.path.getsize("temp.p")/1e6)
- os.remove('temp.p')
-
-print_size_of_model(model)
-print_size_of_model(quantized_model)
-
-######################################################################
-# Second, we see faster inference time, with no difference in evaluation loss:
-#
-# Note: we set the number of threads to one for single threaded comparison, since quantized
-# models run single threaded.
-
-torch.set_num_threads(1)
-
-def time_model_evaluation(model, test_data):
- s = time.time()
- loss = evaluate(model, test_data)
- elapsed = time.time() - s
- print('''loss: {0:.3f}\nelapsed time (seconds): {1:.1f}'''.format(loss, elapsed))
-
-time_model_evaluation(model, test_data)
-time_model_evaluation(quantized_model, test_data)
-
-######################################################################
-# Running this locally on a MacBook Pro, without quantization, inference takes about 200 seconds,
-# and with quantization it takes just about 100 seconds.
-#
-# Conclusion
-# ----------
-#
-# Dynamic quantization can be an easy way to reduce model size while only
-# having a limited effect on accuracy.
-#
-# Thanks for reading! As always, we welcome any feedback, so please create an issue
-# `here `_ if you have any.
diff --git a/advanced_source/extend_dispatcher.rst b/advanced_source/extend_dispatcher.rst
index f3ae1e7e559..12f15355f5f 100644
--- a/advanced_source/extend_dispatcher.rst
+++ b/advanced_source/extend_dispatcher.rst
@@ -17,7 +17,7 @@ to `register a dispatched operator in C++ `_ and how to write a
What's a new backend?
---------------------
-Adding a new backend to PyTorch requires a lot of developement and maintainence from backend extenders.
+Adding a new backend to PyTorch requires a lot of development and maintenance from backend extenders.
Before adding a new backend, let's first consider a few common use cases and recommended solutions for them:
* If you have new algorithms for an existing PyTorch operator, send a PR to PyTorch.
@@ -30,7 +30,7 @@ Before adding a new backend, let's first consider a few common use cases and rec
In this tutorial we'll mainly focus on adding a new out-of-tree device below. Adding out-of-tree support
for a different tensor layout might share many common steps with devices, but we haven't seen an example of
-such integrations yet so it might require addtional work from PyTorch to support it.
+such integrations yet so it might require additional work from PyTorch to support it.
Get a dispatch key for your backend
-----------------------------------
@@ -67,12 +67,12 @@ To create a Tensor on ``PrivateUse1`` backend, you need to set dispatch key in `
Note that ``TensorImpl`` class above assumes your Tensor is backed by a storage like CPU/CUDA. We also
provide ``OpaqueTensorImpl`` for backends without a storage. And you might need to tweak/override certain
methods to fit your customized hardware.
-One example in pytorch repo is `Vulkan TensorImpl `_.
+One example in pytorch repo is `Vulkan TensorImpl `_.
.. note::
Once the prototype is done and you plan to do regular releases for your backend extension, please feel free to
- submit a PR to ``pytorch/pytorch`` to reserve a dedicated dispath key for your backend.
+ submit a PR to ``pytorch/pytorch`` to reserve a dedicated dispatch key for your backend.
Get the full list of PyTorch operators
@@ -361,7 +361,7 @@ actively working on might improve the experience in the future:
* Improve test coverage of generic testing framework.
* Improve ``Math`` kernel coverage and more comprehensive tests to make sure ``Math``
- kernel bahavior matches other backends like ``CPU/CUDA``.
+ kernel behavior matches other backends like ``CPU/CUDA``.
* Refactor ``RegistrationDeclarations.h`` to carry the minimal information and reuse
PyTorch's codegen as much as possible.
* Support a backend fallback kernel to automatic convert inputs to CPU and convert the
diff --git a/advanced_source/generic_join.rst b/advanced_source/generic_join.rst
index 30259650278..0fb0d5528d2 100644
--- a/advanced_source/generic_join.rst
+++ b/advanced_source/generic_join.rst
@@ -369,7 +369,7 @@ of inputs across all ranks.
def join_hook(self, **kwargs) -> JoinHook:
r"""
Return a join hook that shadows the all-reduce in :meth:`__call__`.
-
+
This join hook supports the following keyword arguments:
sync_max_count (bool, optional): whether to synchronize the maximum
count across all ranks once all ranks join; default is ``False``.
@@ -446,5 +446,5 @@ Some key points to highlight:
.. _Getting Started with Distributed Data Parallel - Basic Use Case: https://pytorch.org/tutorials/intermediate/ddp_tutorial.html#basic-use-case
.. _Shard Optimizer States with ZeroRedundancyOptimizer: https://pytorch.org/tutorials/recipes/zero_redundancy_optimizer.html
.. _DistributedDataParallel: https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html
-.. _join(): https://pytorch.org/docs/stable/_modules/torch/nn/parallel/distributed.html#DistributedDataParallel.join
+.. _join(): https://docs.pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html#torch.nn.parallel.DistributedDataParallel.join
.. _ZeroRedundancyOptimizer: https://pytorch.org/docs/stable/distributed.optim.html
diff --git a/advanced_source/pendulum.py b/advanced_source/pendulum.py
index 38524cfff40..3084fe8312b 100644
--- a/advanced_source/pendulum.py
+++ b/advanced_source/pendulum.py
@@ -33,9 +33,9 @@
In the process, we will touch three crucial components of TorchRL:
-* `environments `__
-* `transforms `__
-* `models (policy and value function) `__
+* `environments `__
+* `transforms `__
+* `models (policy and value function) `__
"""
@@ -384,7 +384,7 @@ def _reset(self, tensordict):
# convenient shortcuts to the content of the output and input spec containers.
#
# TorchRL offers multiple :class:`~torchrl.data.TensorSpec`
-# `subclasses `_ to
+# `subclasses `_ to
# encode the environment's input and output characteristics.
#
# Specs shape
@@ -604,7 +604,7 @@ def __init__(self, td_params=None, seed=None, device="cpu"):
env,
# ``Unsqueeze`` the observations that we will concatenate
UnsqueezeTransform(
- unsqueeze_dim=-1,
+ dim=-1,
in_keys=["th", "thdot"],
in_keys_inv=["th", "thdot"],
),
diff --git a/advanced_source/python_custom_ops.py b/advanced_source/python_custom_ops.py
index 9111e1f43f4..1f20125f785 100644
--- a/advanced_source/python_custom_ops.py
+++ b/advanced_source/python_custom_ops.py
@@ -3,7 +3,7 @@
"""
.. _python-custom-ops-tutorial:
-Python Custom Operators
+Custom Python Operators
=======================
.. grid:: 2
@@ -30,6 +30,12 @@
into the function).
- Adding training support to an arbitrary Python function
+Use :func:`torch.library.custom_op` to create Python custom operators.
+Use the C++ ``TORCH_LIBRARY`` APIs to create C++ custom operators (these
+work in Python-less environments).
+See the `Custom Operators Landing Page `_
+for more details.
+
Please note that if your operation can be expressed as a composition of
existing PyTorch operators, then there is usually no need to use the custom operator
API -- everything (for example ``torch.compile``, training support) should
@@ -66,7 +72,7 @@ def display(img):
######################################################################
# ``crop`` is not handled effectively out-of-the-box by
# ``torch.compile``: ``torch.compile`` induces a
-# `"graph break" `_
+# `"graph break" `_
# on functions it is unable to handle and graph breaks are bad for performance.
# The following code demonstrates this by raising an error
# (``torch.compile`` with ``fullgraph=True`` raises an error if a
@@ -85,9 +91,9 @@ def f(img):
#
# 1. wrap the function into a PyTorch custom operator.
# 2. add a "``FakeTensor`` kernel" (aka "meta kernel") to the operator.
-# Given the metadata (e.g. shapes)
-# of the input Tensors, this function says how to compute the metadata
-# of the output Tensor(s).
+# Given some ``FakeTensors`` inputs (dummy Tensors that don't have storage),
+# this function should return dummy Tensors of your choice with the correct
+# Tensor metadata (shape/strides/``dtype``/device).
from typing import Sequence
@@ -106,7 +112,10 @@ def crop(pic: torch.Tensor, box: Sequence[int]) -> torch.Tensor:
def _(pic, box):
channels = pic.shape[0]
x0, y0, x1, y1 = box
- return pic.new_empty(channels, y1 - y0, x1 - x0)
+ result = pic.new_empty(y1 - y0, x1 - x0, channels).permute(2, 0, 1)
+ # The result should have the same metadata (shape/strides/``dtype``/device)
+ # as running the ``crop`` function above.
+ return result
######################################################################
# After this, ``crop`` now works without graph breaks:
@@ -130,6 +139,11 @@ def f(img):
# ``autograd.Function`` with PyTorch operator registration APIs can lead to (and
# has led to) silent incorrectness when composed with ``torch.compile``.
#
+# If you don't need training support, there is no need to use
+# ``torch.library.register_autograd``.
+# If you end up training with a ``custom_op`` that doesn't have an autograd
+# registration, we'll raise an error message.
+#
# The gradient formula for ``crop`` is essentially ``PIL.paste`` (we'll leave the
# derivation as an exercise to the reader). Let's first wrap ``paste`` into a
# custom operator:
@@ -203,7 +217,7 @@ def setup_context(ctx, inputs, output):
######################################################################
# Mutable Python Custom operators
# -------------------------------
-# You can also wrap a Python function that mutates its inputs into a custom
+# You can also wrap a Python function that mutates its inputs into a custom
# operator.
# Functions that mutate inputs are common because that is how many low-level
# kernels are written; for example, a kernel that computes ``sin`` may take in
@@ -260,5 +274,5 @@ def f(x):
# For more detailed information, see:
#
# - `the torch.library documentation `_
-# - `the Custom Operators Manual `_
+# - `the Custom Operators Manual `_
#
diff --git a/advanced_source/semi_structured_sparse.py b/advanced_source/semi_structured_sparse.py
index 38c2c6878b3..e4bca79b9af 100644
--- a/advanced_source/semi_structured_sparse.py
+++ b/advanced_source/semi_structured_sparse.py
@@ -43,6 +43,8 @@
# - A NVIDIA GPU with semi-structured sparsity support (Compute
# Capability 8.0+).
#
+# .. note:: This tutorial is tested on an NVIDIA A100 80GB GPU. You may not see similar speedups on newer GPU architectures, For the latest information on semi-structured sparsity support, please refer to the README `here
+#
# This tutorial is designed for beginners to semi-structured sparsity and
# sparsity in general. For users with existing 2:4 sparse models,
# accelerating ``nn.Linear`` layers for inference with
@@ -52,7 +54,6 @@
import torch
from torch.sparse import to_sparse_semi_structured, SparseSemiStructuredTensor
from torch.utils.benchmark import Timer
-SparseSemiStructuredTensor._FORCE_CUTLASS = True
# mask Linear weight to be 2:4 sparse
mask = torch.Tensor([0, 0, 1, 1]).tile((3072, 2560)).cuda().bool()
@@ -207,9 +208,10 @@
import transformers
# force CUTLASS use if ``cuSPARSELt`` is not available
-SparseSemiStructuredTensor._FORCE_CUTLASS = True
torch.manual_seed(100)
+# Set default device to "cuda:0"
+torch.set_default_device(torch.device("cuda:0" if torch.cuda.is_available() else "cpu"))
######################################################################
# We’ll also need to define some helper functions that are specific to the
diff --git a/advanced_source/static_quantization_tutorial.rst b/advanced_source/static_quantization_tutorial.rst
deleted file mode 100644
index 3b818aa03aa..00000000000
--- a/advanced_source/static_quantization_tutorial.rst
+++ /dev/null
@@ -1,635 +0,0 @@
-(beta) Static Quantization with Eager Mode in PyTorch
-=========================================================
-**Author**: `Raghuraman Krishnamoorthi `_
-**Edited by**: `Seth Weidman `_
-
-This tutorial shows how to do post-training static quantization, as well as illustrating
-two more advanced techniques - per-channel quantization and quantization-aware training -
-to further improve the model's accuracy. Note that quantization is currently only supported
-for CPUs, so we will not be utilizing GPUs / CUDA in this tutorial.
-By the end of this tutorial, you will see how quantization in PyTorch can result in
-significant decreases in model size while increasing speed. Furthermore, you'll see how
-to easily apply some advanced quantization techniques shown
-`here `_ so that your quantized models take much less
-of an accuracy hit than they would otherwise.
-Warning: we use a lot of boilerplate code from other PyTorch repos to, for example,
-define the ``MobileNetV2`` model architecture, define data loaders, and so on. We of course
-encourage you to read it; but if you want to get to the quantization features, feel free
-to skip to the "4. Post-training static quantization" section.
-We'll start by doing the necessary imports:
-
-.. code:: python
-
- import os
- import sys
- import time
- import numpy as np
-
- import torch
- import torch.nn as nn
- from torch.utils.data import DataLoader
-
- import torchvision
- from torchvision import datasets
- import torchvision.transforms as transforms
-
- # Set up warnings
- import warnings
- warnings.filterwarnings(
- action='ignore',
- category=DeprecationWarning,
- module=r'.*'
- )
- warnings.filterwarnings(
- action='default',
- module=r'torch.ao.quantization'
- )
-
- # Specify random seed for repeatable results
- torch.manual_seed(191009)
-
-1. Model architecture
----------------------
-
-We first define the MobileNetV2 model architecture, with several notable modifications
-to enable quantization:
-
-- Replacing addition with ``nn.quantized.FloatFunctional``
-- Insert ``QuantStub`` and ``DeQuantStub`` at the beginning and end of the network.
-- Replace ReLU6 with ReLU
-
-Note: this code is taken from
-`here