| « A way of loading Lua scripts | No naked assembly! » |
According to my first ArcanumAlive related thread, it's been two years and six months since I started ArcanumAlive project. Because I had no website created for the project, I created a thread ArcanumClub.ru forums and used it as place for posting updates and reporting any progress of it's development.
That time, I had already something to show, it was a simple game engine, that was able to load terrain from Arcanum's sectors, and then render it, so I posted a few screenshots of the rendered terrain with hope that it would attract someone who could offer some help with the project.
Later, I did renamed the project to OpenArcanum, since I thought that it would perfectly describe it's purpose, and used ArcanumAlive as name for it's parent project, which would bring more projects like OpenArcanum.
While developing OpenArcanum, I made a lot of decisions that slowed down it's progress. Some of them were bad, good or necessary, but it didn't matter, since all of them slowed down the development. And reason? Lack of planning? Lack of experience or skills? Definitely!
When I started with development, I chose object pascal (Delphi) as programming language, since it was the only language I was experienced with. Even I could choose to learn C++ and use it instead of pascal, I decided to not, since it would slow me down more that I could afford, and I wanted to have working game prototype as soon as possible. Also I chose to use DirectX for rendering, because it's feature rich, and that time, I had no idea about cross-platformability, so I had no reason to go for OpenGL.
Few months later, after some discussions with other members of the ArcanumClub community, I found out that it was a bad decision to write OpenArcanum in pascal, since everybody was coding in C++ and using OpenGL, and because of that, nobody wanted (or was able) to contribute to the project. The idea that I would work on OpenArcanum alone forever made me rethink several things. Before I could let somebody else join the development, I had to switch to C++, rewrite everything from pascal, which was very painful task, and took me a couple months to complete.
After that, I had to make another decision. Some folks recommended to me to use Qt SDK to make the code cross-platform, some other folks recommended SDL library and few other third-party libraries. I chose to use Qt SDK, even I didn't liked the idea, because Qt is kinda heavy, very huge in size, and hello world style applications takes tens of megabytes.
I started using Qt in large scale, and replaced all STL stuff with Qt's alternatives. However, three months later, the feeling about Qt got stronger and the size of main executable got bigger, so I decided to stop using Qt. For a while, I was using SDL library, but I didn't liked the API design and it's license, so I stopped using it as well.
I decided to rather write everything by myself, than using something I don't like to use. A huge mistake, I must say, since the development has changed from game to engine, and then to framework development, so I wasn't developing game nor engine anymore.
The size of the project grew from 10 000 to over 100 000 sloc this year, everything written only by myself, and yet the OpenArcanum can't be considered to be a game. The only thing that it does is loading and rendering Arcanum maps, with static and mobile objects, and some scenery animations, but that's all, so it's not a playable game yet.
Luckily, the framework can be now considered to be finished, even it currently works on win32 only, all the code was written in way that allows to easily port it to other platforms, without need for major changes, so it shouldn't be a problem in future.
With a two year delay, I can finally start developing a game I was always intending to make, the OpenArcanum.
12 comments
> Qt is kinda heavy, very huge in size, and hello world style applications takes tens of megabytes.Qt is 10M tops, and it's nothing nowadays. Supposed you don't restrict yourself to writing helloworlds only :)
Huh :) Only qtScribe has over 16MB, and it's core and gui module only. Also, we're not counting in those tens of dynamic libraries that needs to be linked by every module :)The final executable size in memory can be over 50-100MB. The best way how to calculate it would be dumping executable into file while it's running, but that won't be accurate enough, since it can also allocate some dynamic memory after binding, which sometimes can't be even seen thru task manager.
Anyway, the OpenArcanum's executable size is currenly a bit over 5MB, and it doesn't use any static library nor STL (so it's size of my code only), and third-party dynamic libraries (FFMPEG, OpenAL, etc.) have over 10 MB.
If I used Qt, I will have to use a lot of it's modules, and the OpenArcanum's binaries would surely grow over 100 MB.
I want to keep OpenArcanum's memory requirements as low as possible, no matter that we're living in the future, so it won't consume much more memory than Arcanum does.
I believe that you're aims are similar, and AFAIR except me, you want to make it working on consoles and other non-pc platforms, so you probably should watch A:R's memory requirements as well.
You didn't changed your aims in this aspect, did you? :P
See you around ;)
> qtScribe has over 16MBIt's 1.9M.
> The final executable size in memory can be over 50-100MB
It's virtual memory reserved, actually.
> If I used Qt, I will have to use a lot of it's modules, and the OpenArcanum's binaries would surely grow over 100 MB.
Care to prove this your point empirically? :)
Qt is already ported to Android, Wiz, whatsoever. I do hope to have an easy way to port anything based on Qt.
> You didn't changed your aims in this aspect, did you? :P
Why should i? :)
> It's 1.9M.Thinking about Linux only? Have a look at this: http://i54.tinypic.com/256fz46.png
> Care to prove this your point empirically? :)
I can't, but do you know any game that uses Qt? If it's ported to win32, I can measure how much RAM does all the dynamic libraries takes. Also have a look at this pic: http://i51.tinypic.com/vxc360.png
Lots of libraries, and showing only raw size.
Anyway, let's not waste our time with this qt-good-or-not discussion :) We will see if Qt is good for games or not, after A:R gets released. If it proves that my opinion about Qt is corrupted, I'll throw VS away, and start developing with Qt only.
See you later ;)
>Thinking about Linux only?Should i think of something backwards? Linux in no way perfect, anything worse than it not even worth thinking.
Anyway, do "strip" or whatever's on win.
>I can't, but do you know any game that uses Qt?
Tons of. http://qt-apps.org/index.php?xcontentmode=4250x4251x4252x4253x4254 for one. Rift MMORPG comes to mind as well.
>If it's ported to win32, I can measure how much RAM does all the dynamic libraries takes.
No, you can't. You can only check for memory _reserved_ as in "hey, system, i have a suspicion i'll maybe need 1G of RAM - mkay, noted". That's dynamic memory for you.
>Also have a look at this pic: http://i51.tinypic.com/vxc360.png
So?
>If it proves that my opinion about Qt is corrupted, I'll throw VS away, and start developing with Qt only.
No, you won't. You'll find another excuse not to learn something new :)
>Anyway, let's not waste our time with this qt-good-or-not discussion :)
Ok, sometimes it's just fun to probe non-believers with a stick :D
Thanks for the list, but I was thinking about games more like Arcanum or similar, not a tetris :) Rift looks good though.>No, you can't. You can only check for memory _reserved_ as in "hey, >system, i have a suspicion i'll maybe need 1G of RAM - okay, noted".>That's dynamic memory for you.
On win32, you can measure exact memory usage of any process. When you reserve memory (function VirtualAlloc, flag MEM_RESERVE, no other way), it isn't actually allocated nor it increases memory usage of that process until you call VirtualAlloc again within reserved range.
All the memory allocators uses VirtualAlloc on Windows, and you can simply check it by looking into import table of the executable or by hooking the function while the executable is running.
If you mean it in way that memory allocators allocates (not reserves) large memory blocks and then chop from them some space for small memory allocations, then you can also exactly measure memory usage of the process.
You simply hook VirtualAlloc, fill all allocated blocks with your own data and then scan them, while application is running. All bytes that differs in the block were allocated. To make it accurate, you must also hook "free" function, so when the memory gets deallocated, you fill it with your own data again. If the application is written in C/C++, and is using standard malloc/free or new/delete, then you can easily find the functions in the executable and hook them, or you can also easily scan the blocks without need to fill the them with your own data, since the the blocks contains linked list, where first 4 bytes are used for it's meta data, so you can easily iterate thru the block and check out how many bytes are in use.
Simpler way would be just finding the memory allocators granularity, which is usually 64KB, then you can easily round the memory usage to it, without need to hook anything. But that won't be exact usage, obviously, but I wasn't talking about exact memory usage, just rounded to it's granularity. That would be accurate enough, since all unused blocks are released by the allocator when not used (for small allocations) anymore.
That way the external way, internally, memory allocators usually provide some function to report exact memory usage (not just reserved). Profilers are also able to find this function in the executable and report exact memory usage without need to hook anything as well.
Notice how many times I used word easily :)
Btw, which memory allocator are you (is Qt) using? I would suggest using nedmalloc instead of buildin allocator, since it's the fastest memory allocator available, especially for multithreaded environment. After I switched to it, it made things freakishly fast, especially with these settings:#define USE_LOCKS 1 // enable multi threading support
#define NO_SEGMENT_TRAVERSAL 1
#define DEFAULT_GRANULARITY (1024 * 1024 * 16)
#define DEFAULT_TRIM_THRESHOLD (1024 * 1024 * 64)
#define MALLOC_ALIGNMENT 16 // in case you're using SSE SIMD
#include <nedmalloc.c>
Cheers :)
> If you mean it in way that memory allocators allocates (not reserves) large memory blocks and then chop from them some space for small memory allocations, then you can also exactly measure memory usage of the process.No, you can't. You can check for memory allocated, not really used. So i.e. Qt looks if your system got tons of RAM and goes like "8G, cool, let me allocate 256M for file IO buffers", and the very same Qt on Android goes "hmm, 512M tops, let me only use 1M for file IO".
That's the beauty of it, actually: you don't have to worry about anything but first-hand tasks up to the point of final tweaks.
>Btw, which memory allocator are you (is Qt) using?
Built-in, it's fast enough for the time being (for me).
That's actually pretty ugly feature, not nice, since all applications written in Qt uses a lot more memory than they really requires.I just found way how to determine "real" amount of memory needed for qtScribe to run, it's roughly 700KB, but it's mem allocator "reserves" next 27MB for those "buffers". (I have 4GB of RAM, and 2GB of VM on my rig.)
I did also tested if it gives the "reserved" memory back to the system, when it's out of memory. Result: No, it just let the other applications use slow virtual memory. The "reserved" memory is freed only when application gets minimized.
Conclusion: Qt's greedy :) Good luck with that, since Arcanum's memory requirements are only 32MB :P
Cheers.
No, it's nice feature, 'cause any piece of modern day software uses more resources than minimum possible. The thing is to spend those extra to programmer's comfort.Again: nobody cares. And it's 32M for original Arcanum. Good luck packing all the extra features planned into the very same 32M :)
Well, if it were a nice feature, it would implemented as core feature in other languages and frameworks, and instead of that, they have a garbage collector. It isn't nice to allocate so much memory, and naively hope that it will be used in near time, while it isn't used for anything.How you can say that nobody cares? you speak for everyone? I believe that there will be a lot of people (especially those non-pc) who will care a lot about A:R's system requirements. If every Qt's module/cache/etc, reserves so much memory for nothing, then a lot of memory gets wasted for nothing, and those people will notice ;)
Maybe you're right, and I'm too minimalistic, but that doesn't matter, because there will be always people like me, who would whine and start flamewars about it :P
Anyway, I said that it's 32MB, but that number was determined by distributor, normally Arcanum needs at least 66MB of memory (task manager would say exact number) to start a new game.
OpenArcanum won't require more than 128MB with the new features, currently it requires only 54MB for playing, but that's without sounds and music.
See you around.
Funny thing is it's actually IS implemented in quite a few systems. What comes to mind is Windows Vista/7 and modern Linux (tho Linux can be tweaked however you want). And greedy memory managers could pretty well co-live with GCs (as JVM or Parrot prove). What i'm saying is greedy algorithms not necessarily a bad thing, and "premature optimization is the root of all evil".Oh, i do speak for everyone, i definitely wouldn't dare to speak otherwise :)
Comment feed for this post
