Heroes of Might and Magic Community
visiting hero! Register | Today's Posts | Games | Search! | FAQ/Rules | AvatarList | MemberList | Profile


Age of Heroes Headlines:  
5 Oct 2016: Heroes VII development comes to an end.. - read more
6 Aug 2016: Troubled Heroes VII Expansion Release - read more
26 Apr 2016: Heroes VII XPack - Trial by Fire - Coming out in June! - read more
17 Apr 2016: Global Alternative Creatures MOD for H7 after 1.8 Patch! - read more
7 Mar 2016: Romero launches a Piano Sonata Album Kickstarter! - read more
19 Feb 2016: Heroes 5.5 RC6, Heroes VII patch 1.7 are out! - read more
13 Jan 2016: Horn of the Abyss 1.4 Available for Download! - read more
17 Dec 2015: Heroes 5.5 update, 1.6 out for H7 - read more
23 Nov 2015: H7 1.4 & 1.5 patches Released - read more
31 Oct 2015: First H7 patches are out, End of DoC development - read more
5 Oct 2016: Heroes VII development comes to an end.. - read more
[X] Remove Ads
LOGIN:     Username:     Password:         [ Register ]
HOMM1: info forum | HOMM2: info forum | HOMM3: info mods forum | HOMM4: info CTG forum | HOMM5: info mods forum | MMH6: wiki forum | MMH7: wiki forum
Heroes Community > Library of Enlightenment > Thread: On the internals of offered skills when leveling-up a hero.
Thread: On the internals of offered skills when leveling-up a hero. This thread is 17 pages long: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 · «PREV / NEXT»
maretti
maretti


Responsible
Supreme Hero
posted July 24, 2007 09:24 PM

@Dimis: I would really like to help you out but im affraid you lost me long time ago. All I can say is, it would be nice to look at the heroes developement when you pick prefered skills. Something like the skillgroups i mentioned earlier. I have no idea exactly how it could be done.

@Xarfax: At least your posts have some entertaining value, but come on foo. Everyone can claim they know this and that without revealing how it works. You want us to believe you found out something great, but the fact is your knowledge about skilldevelopement isnt that impressive. You are far behind Dimis and Ecoris. All you wanna do is put some doubt in peoples minds (you dont fool me )

I.e.
Quote:
Then i said that you can always get earth with Hack if you want. Everyone said this is just blah, but i proofed them wrong.


You were right, but for the wrong reasons. You didnt prove anyone wrong:

Quote:
you get always offered a magic skill at levl4 when you play a might. After this you get ALWAYS offered a magic skill every 6 level, no matter if you take the first or not.


____________
Crag rules, Orrin and Ivor suck

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Xarfax111
Xarfax111


Bad-mannered
Supreme Hero
The last hero standing
posted July 25, 2007 12:39 AM

Quote:
@Xarfax: At least your posts have some entertaining value, but come on foo. Everyone can claim they know this and that without revealing how it works. You want us to believe you found out something great, but the fact is your knowledge about skilldevelopement isnt that impressive. You are far behind Dimis and Ecoris. All you wanna do is put some doubt in peoples minds (you dont fool me )


I would call that a weak attempt to lure me into giving the proof. You dont have to believe in it. I agree with you that Dimis and Ecoris are far more into statistics then me, but to know how to calculate, doesnt bring you automatically to the results. You know that.

Quote:
you get always offered a magic skill at levl4 when you play a might. After this you get ALWAYS offered a magic skill every 6 level, no matter if you take the first or not.




I dont know where you get this statement from. This statement is correct. If you take all offered magic skills youll get 100% earth. As simple as that.

Of course i proof anyone wrong. Everybody was telling that the skill table for the txt-file show the probabilities. Which they dont. There are other mechanism in the background like "always wisdom" and "always magic skills" and some others. So those probabilities SIMPLY CANT be the correct ones.

Examples:

Earth = probbility 3 out of 112 = plain wrong

@angelito: Im not jealous about anything...thats just plain stupid to say. That would mean that i care about QPs, which i obviously dont.
____________

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted July 25, 2007 01:05 AM
Edited by dimis at 01:09, 25 Jul 2007.

my view

Quote:
I dont know where you get this statement from. This statement is correct. If you take all offered magic skills youll get 100% earth. As simple as that.
Now you are mixing again truth and false. But I am sure this is not done on purpose. Can we forget it though, because this might keep us back from progress?

Quote:
Of course i proof anyone wrong. Everybody was telling that the skill table for the txt-file show the probabilities. Which they dont.
Both correct and incorrect again. If you happen not to be in an "exception level", then these values show probabilities. As Russ (btw, where is he?) pointed out by that time (for the example you give below) it is 3 / sum(weights of NON-accepted skills), so in a sense it shows something like the probability. After all, non-exception levels occur more frequently.

Quote:
There are other mechanism in the background like "always wisdom" and "always magic skills" and some others.
Of course! These mechanisms make the problem so interesting. Btw, it was a way for the designers of the game to "force" somehow the users to take some skills more often than not. Just look on the impact of those mechanisms when you compare the two tables
Always Right
Always Left
Chances for magic schools increase A LOT when you switch from AlwaysRight to AlwaysLeft! While chances for the rest (if not all - can't be sure till I double check and justify my thought) of the skills fall ...
But I still don't accept that there are other mechanisms unknown simply because someone says so. I hope you understand that. As I already stated, the indications are favoring me (I am not seeing this as a personal debate, but I don't know how to say it differently!) at the moment because I have some, while from your side there is nothing more but your word for that. Is this reasoning?

Quote:
Examples:

Earth = probbility 3 out of 112 = plain wrong
I believe we are all saying the same thing but look it from a different perspective.


____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Binabik
Binabik


Responsible
Legendary Hero
posted July 25, 2007 03:23 AM

Quote:
Russ (btw, where is he?)
Same thing that happened to Legend. He has a new girlfriend. nuff said


 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted July 25, 2007 01:22 PM
Edited by dimis at 15:26, 29 Jul 2007.

Preliminaries / Background (Part II)

Initially, I wanted to place this post in page 1 of this thread, so that I can keep all related information in one point. However, it seems I am not allowed to edit that post since more than 1 year has passed from my starting post there (not from my last edit there as you can check at the end of the post! ). If this is not much of a burden, angelito, I would like to get permission somehow to add these info there (assuming a renewal of editing time, since I might need to add some extra info later on); otherwise we can leave it here.
Moreover, in case I need a permission to make the very first post of this thread able to "hold" a table of contents, I request permission for that too. The planned addition follows:

ADDED: Jul 25, 2007.
Reinforcement Learning
Part of some algorithms and the methods used by various programs that are developed in this thread in order to counter the "Skill Advancing Problem" are based on the theory and the advances that are made in a branch of AI called Reinforcement Learning.
What is Reinforcement Learning ? This can be a good starting point to give you a very rough overview of the objects studied in this field.
The book that I used in the past (and I highly recommend it to anyone who wants to work on Machine Learning, or simply wants to realize how some things work in Artificial Intelligence in a very comprehendable language - try it out and tell me if I am wrong!) is that of Richard S. Sutton and Andrew G. Barto Reinforcement Learning: An Introduction which is also available online.  Another good starting pointer is the FAQ section of the book which also gives the brief description about Reinforcement learning in the link above. If you want to have a look on the fascinating applications of Reinforcement Learning don't forget to spend some time in the Case Studies section of the book. If not anything else, please spend some time reading about TD-Gammon (1), (2) which changed dramatically the way modern grand-masters in backgammon play their games. In fact, the successor of TD-Gammon is now banned from all tournaments, and is purely used as a "judge" when people want to analyze certain positions in back-gammon. It is simply unbeatable. Read the stories (and leave math-related paragraphs aside for starters) they are worth it! Of course you can find more information online with your favorite search engine. keyword: "TD-Gammon"

Quote:
I'll need this later on, so ...
Definition: Policy is just what the word implies. To put it simply, in the "Skill Advancing Problem", it is the method/mechanism used to decide whether user picks the left or the right offer when advancing a hero to the next level.

At the moment, I am using solely Monte Carlo methods in some of the programs that are developed in this thread. Actually, I am using Monte Carlo methods for an easier problem compared to what they can actually be used. I am using Monte Carlo methods so that I can evaluate various (deterministic or not) users' policies when selecting skills while advancing a hero to a subsequent level (terminology note: compute the rollout of certain decisions); i.e. compute the "appearance" probability each individual skill has when following a specific policy. But Monte Carlo methods can be used for something bigger. One can implement the model, tell the computer his preference on skills let's say (I'll be more technical to this in case I implement it since it is not so trivial as it may sound right now...) and then use that "Monte Carlo Oracle" online; i.e. when actually advancing a hero during the game and yield the best possible decision based on his preference on skills.

Monte Carlo Methods
Again, a very good introduction of the Monte Carlo Methods can be found in the book of A. Barto and R. Sutton online. Another good source point seems to be this one (contents here) if you have captured the general idea about the problems that are solved by Reinforcement Learning Methods. In fact I like so much the overview of the first paragraph in this latter reference so that I will copy-paste it down here:
Quote:
Numerical methods that are known as Monte Carlo methods can be loosely described as statistical simulation methods, where statistical simulation is defined in quite general terms to be any method that utilizes sequences of random numbers to perform the simulation. Monte Carlo methods have been used for centuries, but only in the past several decades has the technique gained the status of a full-fledged numerical method capable of addressing the most complex applications. The name ``Monte Carlo'' was coined by Metropolis (inspired by Ulam's interest in poker) during the Manhattan Project of World War II, because of the similarity of statistical simulation to games of chance, and because the capital of Monaco was a center for gambling and similar pursuits.

Another interesting story (closely related to TD-Gammon and far more popular) is the application of Monte Carlo Methods in the game of Blackjack The book of A. Barto and R. Sutton describe the method and the results vividly in the chapter of Monte Carlo Methods in their book, but don't tell the whole story that accompanies the algorithm and the impact in casino-life. As I said, Monte Carlo Methods compute optimal policies (usually denoted with greek letter pie). So how do you play optimally in Blackjack? Check figure 5.5 here (that is at the end of the page). I don't have good references for the impact of the results presented in figure 5.5 in the link above (since I have to google around a bit), but the general idea was the following (perhaps with some mistakes - I have to recheck ). A mathematician initially wanted to play Blackjack well because he desperately needed some money. Then he applied MC Methods and reached to the results presented in fig. 5.5. He achieved his goal but didn't work on the problem more. This specific policy, although optimal, still works favoring casino slightly in the long run. But, some years later, a group of graduate students at MIT started working on the problem again. And guess what ... they actually found (mathematically proved) that they could beat all casinos in the long run with a fair amount of starting money (of course the policy didn't change, since it was optimal. What really changed is the amount of bets depending on the cards that had already appeared during the game.). So, they started a tour across US and did simply that: collected taxes from casinos! Soon, however, their secret was revealed and they were not allowed to enter again any US-casino. Meanwhile, in order to tackle their system the casinos had some other minor fights with them [as well as with other players who knew the tricks ] such as adding more decks, re-inserting cards in the back of the deck, shuffling cards]. Anyway, this group of 3 people moved then to Europe and started playing. Alas, ... once they entered Monte Carlo (ironic, isn't it?) casino, they were dismissed. Casinos world-wide had formed an alliance and these people simply were not allowed to play! Soon, the casinos forbid any card-counting technique; people who were (and still are) caught to count cards (in order to estimate whether the current game was favoring their side or casino's)  were (are) automatically dismissed from casinos... google around for this story in case it takes me a while to upload some good links. It's very fun and can be profitable too! Blackjack is broken ... Casinos simply don't allow you to exploit the hole with card-counting.
____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Ecoris
Ecoris


Promising
Supreme Hero
posted July 29, 2007 12:18 AM
Edited by Ecoris at 00:21, 29 Jul 2007.

I'm back from a 3-week holiday, and I'm glad to see that this thread hasn't died.

I'm not that interested in asking complicated questions about what the chance that you will be offered skill X or skill Y before level Z. Some people study the skill-offer algorithm in order to be able to valuate the different hero classes better; I do it because I want "to know" how it works.

Having read the posts from the last weeks I feel that it might be good to repeat some of the facts gathered so far:

1. We know how the algorithm works in general and how it uses the odds values.

2. We know of the wisdom and the magic school skill groups and how they work. Futhermore it is very unlikely that other skill groups like these exist. Xarfax wrote:
"There are other mechanism in the background like "always wisdom" and "always magic skills" and some others."
There may very well be some other mechanics (besides the "normal" skill selection of the algorithm and the mentioned skill groups). But they must work in other ways than the two known skill groups.
Those skill groups were discovered through playing and described precisely through testing. Why don't you share your ideas/knowledge with us Xarfax?
"PLUS there is a way to get always Logistics with Hack! (Similar like gaining Earth all the time)."
The way to always get earth relied on the fact that Crag can't learn water magic and that you are offered a magic skill (new or upgrade) at least once every four levels. As long as you always chose the magic skill and kept one skill slot open for earth you would get it soon enough. We all know that (I assume). But "always logistics" must work in a different way. Why don't you tell us how, or at least give us a hint?

3. The "balls in bins" made by dimis also shows with very high probability that there are in fact a very limited number of possible skill trees (I prefer the term skill trees to skill sequences, since sequences was a term used to refer to a vague idea about how the skill-offer algorithm could work).
The number was around 200-250 IIRC.
This relatively low number may explain why tests give different probabilities for skills with the same odds value; it need not be caused by some unknown mechanic.

That's all I can think of at this late hour.
____________

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
pacifist
pacifist


Famous Hero
posted July 29, 2007 11:16 AM

I think you should take skills like pathfinding, scouting and navigation (but this one try to avoid ) if logistics doesn't appear soon. Those skills must be in some group too. It seems to work from my experience but hard to say if I could have had logistics anyway or not. It's hard to give scouting to main hero but in my opinion a main hero doesn't have necessary to have 8 perfect skills . They are grouped in H4 so it's maybe a hint.

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted April 03, 2008 08:52 AM
Edited by dimis at 08:54, 03 Apr 2008.

Do you want to know the future of internals_mc?

If so, refer to the homepage of the program. Version 2 has arrived but unfortunately it does not run under Windows.*

Refer to the Home Log near the end of the homepage, and there you 'll find the source code (which works everywhere but on Windows), as well as a n00b's thoughts on thread-programming under the modifications section. If you have any comments or suggestions about the implementation, these are more than welcome.

I guess this is pretty much it. If I forgot something, let me know.

Cheers,
- dimis -

* In fact, direct porting will be either an easy or an impossible process. I don't expect something in between.

____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted April 08, 2008 07:03 AM

Unleash your cores!

Version 2.0 of internals_mc is now available for Windows.

If you have a quad-core processor report time and specs for cpu for 1,000,000 and 5,000,000 episodes.

Note, that direct comparison regarding the performance of the program with the previous version is unfair, since a different compiler was used. Of course both were MS, and I don't think that the latest compiler optimized many more things, but you never know unless you are the developer of the compiler! lol

In addition to that, under Windows, it seems that threads are treated fairly by the scheduler, and this results in practice (at least from a few sample runs I just made) with small memory consumption. Note though, that in general, you might encounter worst memory requirements with a magnitude 176*n bytes, where n is the number of episodes (simulations). So, just be cautious with that.

Another impact of the above remark, is that for large n (say 5 million), since threads are passed to the cores in an interleaved fashion, at some point many of them finish the work, and eventually main (our sole random number generator) is given priority to feed with random numbers the rest of the threads. This results in some "gaps" which you can view in the Task Manager - cpu monitor. In the future I have to give higher priority to the thread that generates those numbers, or implement the idea of multiple generators working in parallel. At the moment, I didn't do the first one because I don't know how , and I didn't do the latter because I am bored; again .

Anyway,
Have fun by crunching numbers faster!

And I need a report by a quad core!!!


Oh, one final note. Let me know if I should distribute a .dll file. You 'll realize that if the program (exe) does not run.

Regards
- dimis -
____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted April 09, 2008 08:39 PM
Edited by dimis at 20:52, 09 Apr 2008.

By the way ...

Does anybody use ALTP and SPOU skill-selection methods, or am I the only one?
If yes, then I have another question.
Do you like it the way they are? I mean, one probably has to change the preferences file when switching heroes, because you have to have on top 8 of your list the initial skills. Otherwise you get an error message to modify your input files. But in general, the skills that you like to get don't depend on the hero; I mean in the general sense. Of course for individual heroes you have to make some adjustments in your likings because you can not have the same skills with both of them (e.g. one starts with Archery, while another one might not).
So, do you think it is a good idea to automate this process and use one skill-preferences file for all heroes?

Now, switching subject. What about other skill-selection techniques?
We certainly have to implement maretti's idea on groups, a (more than one variation?) method that always accepts Magic schools, and Frank's skill-selection method. Do you have another idea on a skill-selection strategy?

Regarding the "always accept magic" method, some extra details must be determined. Use of a preferences file like in ALTP or SPOU? Use of groups of skills like in "maretti's method"? Something else? Something in between? Both?

I am just asking all these so that I can gradually create a plan countering them late May or early June, but at the same time I am running out of time and the post that hosts internals_mc will soon be locked.

Oh, and by the way, criticism is always welcome. Post any random thought or complaint that you have as a user. It's easier for you than it is for me.

Ooops, yes I have to allow results for intermediate levels as well. I always forget that!
____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
kallen
kallen


Known Hero
posted August 24, 2008 06:32 PM

Hey guys Do you have a ansa magic changes for Magic Heroes??
____________
From the beginnin' to end Losers lose, winners win This is real, we ain't got to pretend The cold world that we in Is full of pressure and pain I thought it would chane
But its stayin' the same

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted August 26, 2008 09:24 AM
Edited by dimis at 09:29, 26 Aug 2008.

Sorry Kallen. There is nothing implemented at the moment for magic heroes. I implemented Frank's skill selection and a variant of that which have not been posted yet, but I want some additional skill selection mechanisms to be implemented. Plus I want to implement some extra features which will allow plotting percentages on a per level basis or graphs on a "per slot" basis. But I have no clue when am I going to do that. It might be in a week. It might be waaaay longer. And I wouldn't bet it's coming very soon.
Of course ppl with programming skills are more than welcome to implement magic heroes as well. That's easier but tedious and boring as is many times the case in programming.
I think a complete/dense coverage of what can be found online (with links/references) for the problem is described here (appeared here).
Thanks for the input.

____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
AlexSpl
AlexSpl


Responsible
Supreme Hero
posted August 26, 2008 12:15 PM

2dimis: Great work! But your program... Does it use statistics only? Because it's no problem to predict skill advancement from the beginning. Just analyze the source.

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted August 26, 2008 10:35 PM

:-)

Be my guest Alex!
I 've never even attempted seriously to reverse engineer even simple programs. So, if you can help us, please share!
____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
AlexSpl
AlexSpl


Responsible
Supreme Hero
posted October 02, 2008 01:04 PM
Edited by AlexSpl at 13:34, 02 Oct 2008.

Skill Advancement Tree



2dimis: I've written a program for HoMM II which builds a skill advancement tree. You can also search for the skill you need badly And, of course, I can write such a program for HoMM III. It's just like HeroScanner for HoMM V.

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Ecoris
Ecoris


Promising
Supreme Hero
posted October 02, 2008 09:30 PM

Interesting. Can it predict the skills a hero in a game will be offered at each level up?
____________

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
AlexSpl
AlexSpl


Responsible
Supreme Hero
posted October 03, 2008 04:05 PM

Yes, it can. Also it can show the minimal number of level-ups to get the skill you need and the way you can get it. And, as a bonus, shows Hero's MP.

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
dimis
dimis


Responsible
Supreme Hero
Digitally signed by FoG
posted October 03, 2008 11:56 PM

I 've no idea what HeroScanner is, but this is interesting if it is implemented for Heroes III.
This would help in each specific game and not just in general.
____________
The empty set

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Ecoris
Ecoris


Promising
Supreme Hero
posted October 04, 2008 11:05 AM

Alex, I would be very interested if you could write a such a program for HoMM III. If you sent me a copy I'd be grateful.
____________

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
AlexSpl
AlexSpl


Responsible
Supreme Hero
posted October 04, 2008 04:34 PM

Today I've finished writing such a program for HoMM III 4.0

You can download it from here

But it is in Russian and without any instruction. Sorry.
Just run. Choose your version of the game. Press "Start" (button with a man on it). Enter Hero's name. Press "Find" (Enter). Wait. Double click on hero's number in the list (for 4.0 only). Secondary skill names in English though.

Would be it easier if I translate my program?

 Send Instant Message | Send E-Mail | View Profile | Quote Reply | Link
Jump To: « Prev Thread . . . Next Thread » This thread is 17 pages long: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 · «PREV / NEXT»
Post New Poll    Post New Topic    Post New Reply

Page compiled in 0.1067 seconds