Vector and Tensor

In Math, according to the wiki, vector space is a collection of objects called vectors, which may be added together and multiplied(scaled) together. the objects can be of different kind such as height, blood pressure, weight etc.

Matrices are a useful notion to encode linear maps, written as a rectangular array of scalars, for example the below m-by-n matrix.

The tensor product V ⊗F W, or simply V ⊗ W, of two vector spaces V and W is one of the central notions of multilinear algebra which deals with extending notions such as linear maps to several variables. A map g : V × W → X is called bilinear if g is linear in both variables v and w. That is to say, for fixed w the map v ↦ g(vw) is linear in the sense above and likewise for fixed v.

The tensor product is a particular vector space that is a universal recipient of bilinear maps g, as follows. It is defined as the vector space consisting of finite (formal) sums of symbols called tensorsv1 ⊗ w1 + v2 ⊗ w2 + … + vn ⊗ wn,

subject to the rulesa · (v ⊗ w) = (a · v) ⊗ w = v ⊗ (a · w), where a is a scalar,(v1 + v2) ⊗ w = v1 ⊗ w + v2 ⊗ w, andv ⊗ (w1 + w2) = v ⊗ w1 + v ⊗ w2. More intuitively, the following diagram suffices:

Sarai provided this simple codes to illustrate the application of vectors in simple neural network computation.

import numpy as np

# sigmoid function
def nonlin(x,deriv=False):
if(deriv==True):
return x*(1-x)
return 1/(1+np.exp(-x))

# input dataset
X = np.array([ [0,0,1],
[0,1,1],
[1,0,1],
[1,1,1] ])

# output dataset
y = np.array([[0,0,1,1]]).T

# seed random numbers to make calculation
# deterministic (just a good practice)
np.random.seed(1)

# initialize weights randomly with mean 0
syn0 = 2*np.random.random((3,1)) – 1

for iter in xrange(10000):

# forward propagation
l0 = X
l1 = nonlin(np.dot(l0,syn0))

# how much did we miss?
l1_error = y – l1

# multiply how much we missed by the
# slope of the sigmoid at the values in l1
l1_delta = l1_error * nonlin(l1,True)

# update weights
syn0 += np.dot(l0.T,l1_delta)

print “Output After Training:”
print l1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.