In Math, according to the wiki, vector space is a collection of objects called vectors, which may be added together and multiplied(scaled) together. the objects can be of different kind such as height, blood pressure, weight etc.
Matrices are a useful notion to encode linear maps, written as a rectangular array of scalars, for example the below m-by-n matrix.
The tensor product V ⊗F W, or simply V ⊗ W, of two vector spaces V and W is one of the central notions of multilinear algebra which deals with extending notions such as linear maps to several variables. A map g : V × W → X is called bilinear if g is linear in both variables v and w. That is to say, for fixed w the map v ↦ g(v, w) is linear in the sense above and likewise for fixed v.
The tensor product is a particular vector space that is a universal recipient of bilinear maps g, as follows. It is defined as the vector space consisting of finite (formal) sums of symbols called tensorsv1 ⊗ w1 + v2 ⊗ w2 + … + vn ⊗ wn,
subject to the rulesa · (v ⊗ w) = (a · v) ⊗ w = v ⊗ (a · w), where a is a scalar,(v1 + v2) ⊗ w = v1 ⊗ w + v2 ⊗ w, andv ⊗ (w1 + w2) = v ⊗ w1 + v ⊗ w2. More intuitively, the following diagram suffices:
Sarai provided this simple codes to illustrate the application of vectors in simple neural network computation.
import numpy as np
# sigmoid function
# input dataset
X = np.array([ [0,0,1],
# output dataset
y = np.array([[0,0,1,1]]).T
# seed random numbers to make calculation
# deterministic (just a good practice)
# initialize weights randomly with mean 0
syn0 = 2*np.random.random((3,1)) – 1
for iter in xrange(10000):
# forward propagation
l0 = X
l1 = nonlin(np.dot(l0,syn0))
# how much did we miss?
l1_error = y – l1
# multiply how much we missed by the
# slope of the sigmoid at the values in l1
l1_delta = l1_error * nonlin(l1,True)
# update weights
syn0 += np.dot(l0.T,l1_delta)
print “Output After Training:”