Step 3: Connecting Functions¶
The code implemented in the previous step
[1]:
import numpy as np
class Variable:
def __init__(self, data):
self.data = data
class Function:
def __call__(self, input):
x = input.data
y = self.forward(x)
output = Variable(y)
return output
def forward(self, x):
raise NotImplementedError()
class Square(Function):
def forward(self, x):
return x ** 2
So far, we’ve created DeZero’s variable and functions. Then, in the previous step, we implemented a Function class called Square that computes the squares. In this step, another new function is implemented and multiple functions are combined to perform a calculation.
3.1 Implementing the Exp function¶
First, we’ll implement one new DeZero function. Here, we implement the calculation \(y = e^x\) (where \(e\) is a Napier number, specifically \(e=2.718...\)). Let’s implement that code right away.
[2]:
class Exp(Function):
def forward(self, x):
return np.exp(x)
As in the case of Square class, it extends Function class and implements the target computation in the forward method. The only difference from the Square class is that the contents of the forward method is changed from x ** 2 to np.exp(x).
3.2 Connecting functions¶
The input and output of the __call__ method of the Function class are both instances of Variable. Therefore, it is natural to use DeZero’s functions in succession. For example, consider the calculation that \(y = (e^{x^2})^2\). In that case, you can write the following code
[3]:
A = Square()
B = Exp()
C = Square()
x = Variable(np.array(0.5))
a = A(x)
b = B(a)
y = C(b)
print(y.data)
1.648721270700128
Here is the code that applies three functions – A, B, and C – in succession. The important thing to note is that the four variables that appear along the way – x, a, b, and y – are all instances of Variable. Since the input and output of the __call__ method of the Function class are unified in the Variable instance, it is possible to apply multiple functions consecutively as described above. Incidentally, the calculations made here can be expressed
as a calculation graph with alternating functions and variables, as shown in Figure 3-1.
Figure 3-1 Computational graph with multiple functions (○ is a variable, □ is a function)
NOTE
As shown in Figure 3-1, a transformation made by sequentially applying multiple functions can be seen as one large function. This function consisting of multiple functions is called a composition function. The important point here is that even if each of the functions that make up the composite function is a simple calculation, if you apply them consecutively, you can do a more complex calculation.
By the way, why do we represent a series of calculations as a “computational graph”? The answer is that we can efficiently find the derivative of each variable (or, more accurately, we are ready to do so). That algorithm is backpropagation. The next step is to extend DeZero so that back-propagation can be achieved.