In Math, according to the wiki, vector space is a collection of objects called vectors, which may be added together and multiplied(scaled) together. the objects can be of different kind such as height, blood pressure, weight etc.

Matrices are a useful notion to encode linear maps, written as a rectangular array of scalars, for example the below m-by-n matrix.

The *tensor product* *V* ⊗_{F} *W*, or simply *V* ⊗ *W*, of two vector spaces *V* and *W* is one of the central notions of multilinear algebra which deals with extending notions such as linear maps to several variables. A map *g* : *V* × *W* → *X* is called bilinear if *g* is linear in both variables **v** and **w**. That is to say, for fixed **w** the map **v** ↦ *g*(**v**, **w**) is linear in the sense above and likewise for fixed **v**.

The tensor product is a particular vector space that is a *universal* recipient of bilinear maps *g*, as follows. It is defined as the vector space consisting of finite (formal) sums of symbols called tensors**v**_{1} ⊗ **w**_{1} + **v**_{2} ⊗ **w**_{2} + … + **v**_{n} ⊗ **w**_{n},

subject to the rules*a* · (**v** ⊗ **w**) = (*a* · **v**) ⊗ **w** = **v** ⊗ (*a* · **w**), where *a* is a scalar,(**v**_{1} + **v**_{2}) ⊗ **w** = **v**_{1} ⊗ **w** + **v**_{2} ⊗ **w**, and**v** ⊗ (**w**_{1} + **w**_{2}) = **v** ⊗ **w**_{1} + **v** ⊗ **w**_{2}. More intuitively, the following diagram suffices:

Sarai provided this simple codes to illustrate the application of vectors in simple neural network computation.

import numpy as np

# sigmoid function

def nonlin(x,deriv=False):

if(deriv==True):

return x*(1-x)

return 1/(1+np.exp(-x))

# input dataset

X = np.array([ [0,0,1],

[0,1,1],

[1,0,1],

[1,1,1] ])

# output dataset

y = np.array([[0,0,1,1]]).T

# seed random numbers to make calculation

# deterministic (just a good practice)

np.random.seed(1)

# initialize weights randomly with mean 0

syn0 = 2*np.random.random((3,1)) – 1

for iter in xrange(10000):

# forward propagation

l0 = X

l1 = nonlin(np.dot(l0,syn0))

# how much did we miss?

l1_error = y – l1

# multiply how much we missed by the

# slope of the sigmoid at the values in l1

l1_delta = l1_error * nonlin(l1,True)

# update weights

syn0 += np.dot(l0.T,l1_delta)

print “Output After Training:”

print l1