Page 6 of 6
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 7:31 am
by Kirby
I'm not an expert on neural nets, but "deep learning" does imply a deeper structure, I.e. more intermediate layers in the network.
There have probably been lots of improvements to the algorithms behind neural networks, but my opinion is that improved computer hardware has also made training neural nets more feasible.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 9:19 am
by John Fairbairn
I have no doubt I'm talking through my backside - but that's form of reverse engineering and it helps me understand.
I talked about "higher structure" (certainly not deeper) but put that badly. The portion of Levitin's work that I leapt on was something higher that
attended to structure.
On the principle that when you are in a hole stop digging, it will best if I just quote Levitin:
Listening to music and attending to its syntactic features - its structure - activated a particular region of the frontal cortex on the left side [of the brain] called pars orbitalis - a sub-section of the region known as Brodmann Area 47. The region we found in our study had some overlap with previous studies of structure in language, but it also had some unique activations. In addition to this left hemisphere activation, we also found activation in an analogous area of the right hemisphere. This told us that attending to structure in music require both halves of the brain, while attending to structure in language only requires the left half.
A little later he says:
We found evidence for the existence of a brain region that processes structure in general, when that structure is conveyed over time.
It is this chronological aspect that makes me wonder whether music and go may significantly share some brain functions.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 1:35 pm
by Kirby
As far as I know, neural networks don't have the concept of left and right brain. I'm not sure what this implies about the learning process.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 1:50 pm
by Tryss
Left and right brain are structure at a wildly different scale than our neural network. A brain has around 100 billions neurons and hundred trillions of synapses.
It's like comparing the urbanism and structure of the whole US east coast metropolis to the urbanism and structure of a small village.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 4:57 pm
by Kirby
There's different scale, but the left/right-ness is also a concept that doesn't exist in neural networks, right?
The human brain has a lot of different areas performing different functions, but a neural network is basically a graph of connected nodes. The intermediate layers are connected to different components, but no explicit functions are designed to match the human brain afaik.
For example, a neural network doesn't have a portion of the graph set aside to simulate the frontal lobe vs the temporal lobe.
Intermediate layers may have different weights after being trained, but this is implicitly learned as opposed to being designed to be similar to sections of the human brain.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 5:31 pm
by Tryss
For example, a neural network doesn't have a portion of the graph set aside to simulate the frontal lobe vs the temporal lobe.
Of course ! But it's kinda like saying there isn't the concept of the suburb and the inner city in my little street.
These brain area are huge. You would need a million interconnected neural network to approach the size of these macro area. And these area have multiple functions.
Our neural network are closer to the organisation of a very small slice of the cortex :
https://en.wikipedia.org/wiki/Cerebral_ ... awings.png
We're at the cellular level, not the organ's one.
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 7:18 pm
by Kirby
Agree. Though, looks like that small village of a neural network is better than our urban human brains at go
Re: Rational choice by amateurs
Posted: Wed Oct 10, 2018 10:43 pm
by yakcyll
Kirby wrote:There's different scale, but the left/right-ness is also a concept that doesn't exist in neural networks, right?
The human brain has a lot of different areas performing different functions, but a neural network is basically a graph of connected nodes. The intermediate layers are connected to different components, but no explicit functions are designed to match the human brain afaik.
For example, a neural network doesn't have a portion of the graph set aside to simulate the frontal lobe vs the temporal lobe.
Intermediate layers may have different weights after being trained, but this is implicitly learned as opposed to being designed to be similar to sections of the human brain.
If you think about structure, then no, there's no such thing as lobes or left and right hemispheres in neural networks. Neural networks aren't supposed to simulate a brain, they are supposed to simulate processes that occur inside of a brain - in this case, transmission of signals between semi-independent neurons. If you consider functionality however, then specific neural networks can be related to specific parts of a brain, e.g. your policy network is responsible for working out sequences and so more related to the left hemisphere, while your value network allows you to evaluate the board on the fly, being more similar to what the right hemisphere is responsible for.
All this talk about comparisons between neural networks and brains is really weird. NNs are pure maths; if you saw the equations for stuff that goes on inside a go bot, you'd have a hard time comparing the two. There's no intuition in there to speak of, just the history of its games concentrated into a couple of numerical computations.
Re: Rational choice by amateurs
Posted: Thu Oct 11, 2018 8:58 am
by Gomoto
you are probably not aware your brain is pure math too
shhh. itsa secret. dont tell
Re: Rational choice by amateurs
Posted: Fri Oct 12, 2018 2:24 am
by paK0
Gomoto wrote:you are probably not aware your brain is pure math too
shhh. itsa secret. dont tell
