Google Git
Sign in
android / platform / external / pytorch / refs/heads/sdk-release / . / torch / csrc / autograd
tree: 87070b557d98ef7791eefaecab39d2905feff5b5 [path history] [tgz]
  1. functions/
  2. utils/
  3. anomaly_mode.cpp
  4. anomaly_mode.h
  5. autograd.cpp
  6. autograd.h
  7. autograd_meta.cpp
  8. autograd_not_implemented_fallback.cpp
  9. autograd_not_implemented_fallback.h
  10. cpp_hook.cpp
  11. cpp_hook.h
  12. custom_function.cpp
  13. custom_function.h
  14. edge.h
  15. engine.cpp
  16. engine.h
  17. forward_grad.cpp
  18. forward_grad.h
  19. function.cpp
  20. function.h
  21. function_hook.h
  22. FunctionsManual.cpp
  23. FunctionsManual.h
  24. grad_mode.h
  25. graph_task.h
  26. InferenceMode.h
  27. init.cpp
  28. input_buffer.cpp
  29. input_buffer.h
  30. input_metadata.cpp
  31. input_metadata.h
  32. jit_decomp_interface.cpp
  33. jit_decomp_interface.h
  34. profiler.h
  35. profiler_kineto.cpp
  36. profiler_kineto.h
  37. profiler_legacy.cpp
  38. profiler_legacy.h
  39. profiler_python.cpp
  40. profiler_python.h
  41. python_anomaly_mode.cpp
  42. python_anomaly_mode.h
  43. python_autograd.h
  44. python_cpp_function.cpp
  45. python_cpp_function.h
  46. python_engine.cpp
  47. python_engine.h
  48. python_enum_tag.h
  49. python_fft_functions.h
  50. python_function.cpp
  51. python_function.h
  52. python_hook.cpp
  53. python_hook.h
  54. python_legacy_variable.cpp
  55. python_legacy_variable.h
  56. python_linalg_functions.h
  57. python_nested_functions.h
  58. python_nested_functions_manual.cpp
  59. python_nn_functions.h
  60. python_saved_variable_hooks.cpp
  61. python_saved_variable_hooks.h
  62. python_sparse_functions.h
  63. python_special_functions.h
  64. python_torch_functions.h
  65. python_torch_functions_manual.cpp
  66. python_variable.cpp
  67. python_variable.h
  68. python_variable_indexing.cpp
  69. python_variable_indexing.h
  70. README.md
  71. record_function_ops.cpp
  72. record_function_ops.h
  73. saved_variable.cpp
  74. saved_variable.h
  75. saved_variable_hooks.h
  76. symbolic.h
  77. TraceTypeManual.cpp
  78. variable.cpp
  79. variable.h
  80. variable_info.cpp
  81. variable_info.h
  82. VariableTypeManual.cpp
  83. VariableTypeUtils.h
torch/csrc/autograd/README.md

Autograd

Autograd is a hotspot for PyTorch performance, so most of the heavy lifting is implemented in C++. This implies that we have to do some shuffling between Python and C++; and in general, we want data to be in a form that is convenient to manipulate from C++.

Our general model is that for any key data type that autograd manipulates, there are two implementations: a C++ type and a Python object type. For example, consider variables in autograd: we have both Variable in variable.h (the C++ type) and THPVariable in python_variable.h (the Python type.) (By the way, THP stands for TorcH Python, not to be confused with THPP, TorcH C++). Variable contains the payload of a variable, while THPVariable just contains a shared_ptr reference to Variable, as well as references to other Python objects which the Python runtime needs to know about. A lot of data accessor implementations in python_variable.cpp simply reach through to the underlying Variable and return the appropriate value.

The most complicated application of this principle is Function, which also supports users implementing custom behavior in Python. We have the following classes:

  • Node in function.h, the C++ type.
  • THPFunction in python_function.h, the Python object type. In python_function.cpp, you can see the boilerplate that tells the Python interpreter about this object.
  • PyNode in python_function.h, a subclass of Node which forwards apply to a Python THPFunction. (NOT a Python object, despite its name!)

Outside of PyNode, the C++ objects largely avoid referencing Python objects (there are a few exceptions, like pyobj in Variable, and PyNode, whose whole point is to let C++ call into Python). And pyobj in Node to ensure uniqueness of the associated python wrapper (if it exists).