My main source for learning is Lizzie. I've been using her to analyze my own games and pro games. Another source is watching bots play each other. So far they seem pretty consistent in their choices, whether AlphaGo, Leela, Elf or other AI engines.

Here's what I have learnt and an overview of what you could call "bot joseki".

I apologize if a similar thread has been created before - I have missed it then.

Modern opening theory as suggested by the bots has the following features:

- taking sente is very important; temperature locally drops fast; it is urgent to play elsewhere
- therefore, contact plays and diagonal plays (either pressing or undermining), are all the rage, since they are more urgent to respond to, hence have more potential to keep sente
- comparatively, approaching from a distance or making an early base have become less common
- since making a base doesn't seem as urgent, pincer patterns are also rather uncommon
- if sente is the first value, next comes territory, which is valued much over influence. Bots don't like influence for its own sake.

This seems to make 4-4 and 3-4 the most popular moves in bot opening, for reasons we will next explore. 4-5 and 3-5 are rarely played. I will say a few words about 3-3

In the middle game, attacking needs to bring profit in terms of (potential territory). Just influence is not good enough. Attacking for the sake of it is simply wrong.

Overall, bots emphasize efficiency of stones. An inefficient configuration punishes itself. Attacking a clumsy group to force it into life can backfire because the opponent can choose to sacrifice it, as it was inefficient all along. Likewise, defending a clumsy group is adding stones to an inefficient configuration.

Towards the endgame, bots obviously calculate the score precisely and further reduce uncertainty by making the victory more certain, often at the expense of the score difference. Humans may prefer forcing the opponent into resignation by making big (but risky) kills, thereby reducing the risk of making a late blunder.

Many thanks for this thread.

Your experience with bots is greater than mine, but I have a couple of different impressions.

First, I am not at all sure that bots value territory much more than outside influence. Admittedly, AlphaGo seems to me to value influence more than Leela. Who knows what the future holds? Certainly, the early 3-3 invasions of the 4-4 suggest that bots favor territory. OTOH, if they do, why do they play the 4-4 so often? They can surely see the early invasions coming.

Second, what I have seen of bots' endgames does not tell me that they are playing safe, although that's the theory. Versus humans, if the human has not resigned, the bots can play almost anything. I think that there is a lot of randomness in such endgames. And when bot faces bot in the endgame, their games are not only close (since neither has resigned) but the positions retain a lot of aji, usually in the form of possible kos, so that it is difficult to play safe.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

I don't think bots are invading because they are greedy for territory. Rather, they do it because they see a way to get out of the sequence cleanly. And I think that's why they don't mind playing 4-4 either - to cleanly get to the next phase.


My reasoning is based on something a pro said at the last U.S. Go Congress (Andy Liu I think, but I'm not positive). He said that the shoulder hit is a favorite tactic of bots because they simplify the game. And he specifically said that you can think of a 3-3 invasion as a shoulder hit from the inside, which helped me understand why bots do what all the old books told us not to do.

My pebbles for the pool:

1. With one major exception, knotwilg's characterisation of the bots' style seemed to marry up with how a pro would talk about ideal play. I therefore inferred that his reference standard in the comparison was perhaps amateur play (or an amateur understanding of pro play).

2. The exception was the putative bot predilection for eschewing bases. It's not something I've noticed but as making a base tends to be gote, the notion would fit in with a predilection for holding on to sente as much as possible.

3. We talk of the 3-3 invasion. I wonder if that's a misnomer (and it's very, very rarely called that in Japanese). In human play I think we tend to assume an invasion will be gote (if it's not, we further assume the invade made a big mistake either before or after the invasion). What's special about the bot 3-3 play was that the invader expects to come out with sente. I'm suggesting therefore that it's not really an invasion and so to talk about it usefully we need to find another term (a spoiler tactic?).

4. The main effect of seeing AI play on me has been to look at pro games with fresh eyes. Far and away the most dominant change in perception has been that Edo games are much more AI-like than I think any of us would have expected, and to me probably much more so than modern pro games (except for Go Seigen). In particular, the number of probes and shoulder hits in the old games seems much higher. (As to why Go might be different, perhaps it is because when he was young his contemporaries in Japan were mainly learning by playing each other in what was then the modern style, but Go was forced to learn from books of old games.)

5. All pros stress efficiency of plays, so much so that they can even dispense with counting the territories and count the inefficiencies instead. But I believe they see efficiency in a different way from even the best amateurs. And I suspect bots see efficiency in a yet different way from pros. Quite what that might be I have no idea, but what I can say is that I think if you look at a finished bot game you cannot tell whether it was played by a bot. There are still clumps of stones, and lines of stones squeezed between enemy stones (i.e. on dame points). The efficient work done by these stones (if it exists) must have been done at a much earlier stage or must be offset by countable inefficiencies by the opponent. We perhaps need to find a new metric for efficiency to understand bot play.

6. I am struck by the continued misuse of influence vs thickness here. It's not enough to respond that we all know what is meant. I would retort: "No, we don't, otherwise we wouldn't be talking about bot influence like this." I have already done a review somewhere of Yokota Shigeaki's pre-AI book (2010) on how to appraise thickness. I was intrigued by it then. Now I am impressed by it. A reminder of a couple of his most salient but novel points: (1) even single stones can have thickness; (2) thickness has a major time element. Strange how those points seem to have been brought to the fore also by AI...

Thanks John for the supportive and well thought comments.



Probably, yes. By definition my analysis amounts to an amateur understanding of pro play (and bot play).




I made a similar observation, that the 3-3 "invasion" is more of an inverse shoulder hit.



I found that to be the most striking similarity between pro and bot play (through the eyes of an amateur).



This is perhaps the only place where I cannot concur, at least for now. The following may unintentionally irritate you but I've always felt that the scholarship about these concepts and responses to amateur attempts to grasp them have been similar to this immortal prank by Jim Florentine https://www.youtube.com/watch?v=QaRtrffvgUg : "No that's not it".
I have actually given up on guessing what it may be and now hold clear mental concepts about influence and strength for myself, which may but likely don't match the mystified concepts by scholars and presumably professionals. They help me understand pro play and bot play, which is the point of concepts. It may be that "a single stone having thickness" and "thickness having a major time element" are useful ways for pros or scholars to think about the concept, but they confuse me more than anything.

Still, those quibbles aside, much appreciated and continue to appreciate your view at things and your response to my attempts to learn and convey.

I concur.



I am unaware of any new concepts spawned by the bots -- yet. But I do think that the bots are forcing pros to reconsider, perhaps even redefine, go concepts, most particularly in the realm of central influence, and also the concept of a base.



As far as I can tell, influence is a Western term introduced around 50 years ago by Zobrist. I do not equate it with atsumi or gaisei. It goes without saying that I do not agree with the treatment on Sensei's Library (See https://senseis.xmp.net/?Influence ). I use it to mean the effect of a stone or stones on other stones and empty points.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Defining "influence" as "the effect of a stone or stones on other stones and empty points" is too trivial for me. Of course a stone has influence on nearby stones and intersections: that's exactly the difference of it being there or not.

Reduced to its most basic treatment, I define influence and thickness as "some sort of wall" and "a wall that is well connected and alive". One derived meaningful insight is that a firmly connected chain forming a 90° angle with two legs that function at influential walls is thick, because it can make an eyespace-forming base in either direction. Another derived insight is that multiple centre facing walls across the board, can cooperate to form whole board thickness. I know there are other meanings of whole board, like the overall absence of threats, but that's more of a cumulative effect of local thickness. Anyway.

These are useful concepts for me to evaluate an exchange in the corner: White has territory, Black has influence.
You can say "well, but that's not what influence really means". Fine by me, but I almost stopped caring, because 1) I'd like to get a name for what I call influence and thickness, since these things help me evaluating a position 2) I'd like to know what influence and thickness mean, if "that's not it", but 3) neither have really happened over the course of the years, we mostly got variations on "no, that's not it" or vague hints as to what it could mean, between esoteric and almost meaningless, which often made me wonder if we were supposed to understand these concepts at all and draw some kind of benefit of them, or just accept that we're mere mortals and have no access to these mystical thoughts professionals have, because we're not born there, don't speak the language, don't have their year long education ... We're at best allowed to speak of it but not pretend to understand it, and even speaking of these mysteries is better left to the high priests who spent a lifetime studying the holy script.

I'll be a heathen then and continue cursing, until I see the light.

In informal parlance I use influence to mean outside influence or central influence. There are many corner fights that end with one side getting territory and the other side getting influence. Using the simpler term, influence, I don't have worry about the niceties of the term, thickness, which, as John Fairbairn repeatedly points out, Western amateurs do not understand. Furthermore, the simpler term allows me to discuss things such as these positions:

Here is a diagram from jlt's and Ian Butler's first jubango game (viewtopic.php?t=16386 ).

Click Here To Show Diagram Code

[go]$$Bc Keima enclosure
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 3 . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 1 . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

is a standard enclosure, one which Go Seigen had no quarrel with in his writings about 21st century go.

Click Here To Show Diagram Code

[go]$$Bc Ogeima enclosure
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 3 . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 1 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . 5 . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

But bots these days prefer this enclosure to the keima enclosure. Using the simple idea of influence, can we not say that this enclosure emphasizes territory less and (outside) influence more?

Click Here To Show Diagram Code

[go]$$Bc Two space high enclosure
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 3 . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 1 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 5 . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . , . . . . . 2 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Bots also like this enclosure, which humans hardly played before the bot era, and which stresses influence even more, and central influence at that. OTOH, it is hardly thick, it is rather thin for an enclosure.

Click Here To Show Diagram Code

[go]$$Wcm8 Old joseki
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 6 5 7 . . |
$$ | . . . X . . . . . X . . . O 8 2 1 . . |
$$ | . . . , . . . . . , . . . 0 . X 3 . . |
$$ | . . X . . . . . . . . . . . . 4 . . . |
$$ | . . . . . . . . . . . . . . . . 9 . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

In the top right corner White gets territory, Black gets influence. But Leela suggests a different move for .

Click Here To Show Diagram Code

[go]$$Wcm8 Leela's play
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . 6 5 7 . . |
$$ | . . . X . . . . . X . . . O 8 2 1 . . |
$$ | . . . , . . . . . , . . . . . X 3 . . |
$$ | . . X . . . . . . . . . . . . 4 . . . |
$$ | . . . . . . . . . . . . . . . . 9 . . |
$$ | . . . . . . . . . . . . . . 0 . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . O . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

Leela suggests this keima. Isn't in the game both more territory oriented and thick, while Leela's play is both influence oriented and somewhat thinner?

Finally, an example from Go Seigen's writings:

Click Here To Show Diagram Code

[go]$$Bc Yonrensei counter
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . 6 . . . . . |
$$ | . . . 2 . . . . . , . . . . . 1 . . . |
$$ | . . . . . . . . . . . . . 8 . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . 5 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . 4 . . . . . 7 . . . . . 3 . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

When makes yonrensei, Go Seigen suggests the one space jump, . Is not influence oriented, yet not thick?

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

A over-simplified explanation for why it is hard to agree on what is influence, what is thickness, how are they different in go is just that they don't really mean anything Rather, it is just the human brain trying to deal with things that are too complex to read ahead and analyze in detail, and therefore we try to put many things in a few large buckets and pretend they are similar.

I agree with Knotwilg: stop caring about semantics is the practical thing to do, I did the same.

The practical thing to do, as I see it, with respect to what people call influence/thickness/strength: does a given position of my opponent have some clear weakness that can be exploited with a proven sequence of moves? Then approach and exploit it. Does it not have any such weakness? Then stay away from it, since getting close with hurt myself.

_________________
Sorin - 361points.com

In all these examples, the difference is made by efficiency rather than it being influence. Of course, deep down, the reason Lizzie's moves are more efficient is that they strike a better balance between territory and influence

Seriously though, when LZ has to make a choice between territory and influence, in the ways that we have felt them to be equally valuable, she will pick territory UNLESS going for influence allows her to take sente, like in the low 3-4 approach, pincer joseki.

But when a move builds territory and influence together, then her strive for efficiency may indeed favor the less cautious, more audacious, less territorial, more influencial, overall more efficient move.

Recently I switched to large enclosures myself. I feel bots like them for similar reason as other decisions:

Reviewing a game often shows a drop in winrate after an exchange. This happens even after deciding between (mutually exclusive) A and B - for playing either. This likely means the bot prefers to keep the choice for the future, preserving options.

In a sense large enclosures are related to hoshis, they can be used multiple ways. One can allow invasion to gain outside strength, but can also opt for their territory conversion later. A small enclosure is less flexible, so more exploitable.

I think this flexibility, the wider range of future options is their real value, not influence itself.

We disagree about the rather than.



Yes, I think that the top bots are better than we now are in striking a balance between competing alternatives.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

With Elf I have noticed the opposite. Maybe which way to play depends upon a case by case basis. There may also be a difference in style between bots. For instance, Sakata preferred to take kikashi early, while Takagawa preferred to wait. Since they played so often against each other, this contrast is a point of interest in their games.



Well, what does influence mean (versus territory)? Influence is less settled. Flexibility is part of it.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

I suppose it may have to do with other developments elsewhere. Making a decision unnecessarily early allows the opponent to exploit it by turning into a bad decision. OC making it too late allows him to respond better/differently, or not at all. This may also be related to sente. Bots like stable positions where they can safely tenuki from at will. Losing options, being committed to a direction means tied hands with more future responsibilities. Sometimes the gains overweight these minuses, sometimes not.

BTW I'm using the unofficial 15b nets (and sometimes Elf2) on a weak computer. Such loss of value from the loss of options is probably more observable in raw network evaluations (which I often spot in the console) than after a lot of playouts. Also I don't think this applies to kikashis, at least I don't remember seeing significant drop for those even where the bot wouldn't play them by itself. But they shouldn't lose real options either.

I typically hear "influence" for outside influence. So it can also mean loss of option of edge territory, and responsibility to prevent damefication.

Some thoughts (I use Leela myself):

1) IMO that we see these early 3-3 plays doesn't mean that bots prefer territory. If so, they would play the 3-3 in the first place. We used to think of this invasion as bad play. So now we see this play and we think "Oh wow, it's better to play there." And we feel kinda cheated from the 4-4 perspective. But the thing is, whereas before the invading side was inferior, now (without hane-connect exchange) the Bots value it as an even result. It's still perfectly fine to get the outside, it's just not overwhelmingly better as before, just equal. This will take some time to get used to

2) My biggest realization from reviewing my games with Leela was the confirmation of the old proverb "Don't use thickness to surround territory!".

Whenever I try to surround in the centre, she strongly disagrees and instead would just take the next big point that's available. In her mind, thickness should just be left, use it to invade somewhere else and such and the stronger it is, the farther you should play away from it, even on the other side of the board. I feel that a general rule might be: "Play where you're weakest!". This comes out in fights a lot, but can also be viewed as a global concept. Of course you can get stronger by making your opponent weaker and the other way round, so this is not always straightforward to apply.

3) In the early opening I think this might summarize Leelas preferences:
1. Take empty corner (3-4 or 4-4)
2. Approach/shimari with 3-4
3. Approach/3-3/extend with 4-4
4. Answer some approach (here it gets more complicated and will depend on the board which direction to play)

4) She loooves to push.

Hmm, so, close to opponent's thickness?

The general rule is rather: "play where you make the biggest difference". A truism, admittedly

As for playing 4-4 which invites an early 3-3 invasion, I can't help but feeling that is still an unstable phase in bot development. I just can't accept that 3-3 is good enough, even urgent for the invader but not for the owner of the 4-4 stone. Now I know that "your opponent's good move is your good move" is not always to be taken literally, rather regionally, but still, there's a big difference between an approach/enclosure move, where we see the equivalence, and the 3-3 invasion, which is a singularity. They are too different in nature to be two sides of the same coin. For example, if making an iron pillar would be as urgent as invading 3-3, then I could accept the equivalene of the moves.

Fair point!



Well as you said it's not a literal meaning. The Bots don't care what it looks like, they just care about the numbers. In the early opening it seems that 1 space low approach 4-4 and 3-3 invasion have almost the same value. So in that sense, it is not hard to imagine that a 1 or 2 space low shimari from 4-4 (which is follow-up if opponent doesn't play that corner) can have the same value as opponent invading at 3-3.

Btw I have seen one type of situation where Leela actually plays 3-3 as hoshi owner, that is as a follow-up if opponent doesn't respond to 1 space low pincer (which it plays sometimes, although not that often).

The fact that bots like the 3-3 invasion early but don't like the 4-4 owner playing any move that early to prevent the 3-3 invasion is a interesting observation. Brainstorming possible human-intuitable ways that this might still be a consistent state of affairs for strong play:

One way this could be is if the 3-3 invasion is both "locally" a very good move (so that as a baseline you are rarely dissatisfied with already making that choice), and also has additional non-local probing value (even when gote!) by forcing the opponent to commit to whether they want a wall facing this way, or that way, or want one of the variations where they share or retake the corner. The idea being that even though you can defer 3-3 and play other opening moves first and the opponent will usually not spend a move to defend and will still let you play 3-3 later, if you do defer it then the opponent might be able to better choose the variation to make your other opening moves less effective. Then it could make sense to play 3-3 even well before the opponent would stop you, forcing your opponent to commit so that you can determine how to develop the sides.

They do like the 3-6 enclosure fairly early, though. Early enough to prevent some 3-3 invasions.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Yes but that is exactly the point I am discussing. "The opponent's move is your good move":

1) corner approach <-> corner enclosure
2) corner invasion <-> corner enclosure ?

Both the approach and the invasion "exist" in bot evaluation. However only the enclosure exists. This is out of balance.