Do you think iTunes is bloated?

[kirkmc]

Yes, I asked the question in a music forum, and the answers are quite different, because all of the people there are serious music fans (classical music listeners with large music collections) for whom music player software is very important.

And you say Go players don't care for music, nice insult but a bit off the mark I think.

You know that's not what I meant...

[CarlJung]

Oh, no, it's not about swapping. On Mac OS X, programs will "hold on" to memory when they need it, then let it go when they don't. Only if they're doing something where they actively need the memory will they need to swap. So iTunes taking up 300 MB, as in imabuddha's case, (right now, it's at 249 MB for me, as I listen to music with a very large library), means nothing. It doesn't mean that memory is locked, frozen or anything, and, unless the app is actively doing a lot, there will be no swapping. Granted, if your computer doesn't have enough RAM - as in Ross's case - then you will have a lot of swapping. But Ross is running an old computer with less RAM than what the OS requires, so I think his case is not a good example.

That sounds... different. Can anyone point me to more info about how this works?

Well, there's this, that gives an overview:

http://www.macosxhints.com/article.php? ... 3140025184

I see. The memory usage includes the OS' dynamic IO cache as well. What was confusing me was the part about applications holding on to memory but at the same time somehow being able to release it when someone else needed it. It's not actually the application that holds on to the memory but the OS. It's the presentation of used memory that makes it look like the memory is allocated by the application since it includes the OS' IO cache as well.

[kirkmc]

I see. The memory usage includes the OS' dynamic IO cache as well. What was confusing me was the part about applications holding on to memory but at the same time somehow being able to release it when someone else needed it. It's not actually the application that holds on to the memory but the OS. It's the presentation of used memory that makes it look like the memory is allocated by the application since it includes the OS' IO cache as well.

Yes, I think it's that apps can say they need to hold not memory, but also say that they don't need specific bit, so the OS can then reclaim it as needed. If you look at ram usage in OS X you can see that, at times, it will look as though it's all being used, and at others, a lot of it suddenly gets freed up. As best as I can understand, you really don't need to worry about ram unless you really don't have enough, or if you really use a lot.

[CarlJung]

I see. The memory usage includes the OS' dynamic IO cache as well. What was confusing me was the part about applications holding on to memory but at the same time somehow being able to release it when someone else needed it. It's not actually the application that holds on to the memory but the OS. It's the presentation of used memory that makes it look like the memory is allocated by the application since it includes the OS' IO cache as well.

Yes, I think it's that apps can say they need to hold not memory, but also say that they don't need specific bit, so the OS can then reclaim it as needed. If you look at ram usage in OS X you can see that, at times, it will look as though it's all being used, and at others, a lot of it suddenly gets freed up. As best as I can understand, you really don't need to worry about ram unless you really don't have enough, or if you really use a lot.

Actually the applications are not aware of the IO caching. It's entirely up to the OS. If an application reads a file and there is enough free RAM, the OS caches it (or part of the file). The next time the application wants to read the same file it will get it from RAM instead which will be much faster. From the application's perspective it is just requesting to read a file. It does it in the same way both times. It's just that for the second read, the OS will decide to fetch it from RAM instead. If another program needs a lot of RAM the cached file will be thrown out from RAM and if it needs to be read again it will be read from disc.

[deja]

I see. The memory usage includes the OS' dynamic IO cache as well. What was confusing me was the part about applications holding on to memory but at the same time somehow being able to release it when someone else needed it. It's not actually the application that holds on to the memory but the OS. It's the presentation of used memory that makes it look like the memory is allocated by the application since it includes the OS' IO cache as well.

An example par excellence of what it means to engage in reasonable and mature discourse. Well done, Carl!

[Aphelion]

Kirkmc, you seem to have your mind long made up about this issue, and you have ready-made excuses on why the opinion of those who disagree with you is marginal and not as valid as yours.

Why are you still trolling for responses here then? Why not just go ahead and write the article the way you clearly want to anyways?

[kirkmc]

Kirkmc, you seem to have your mind long made up about this issue, and you have ready-made excuses on why the opinion of those who disagree with you is marginal and not as valid as yours.

Why are you still trolling for responses here then? Why not just go ahead and write the article the way you clearly want to anyways?

Well, I was trying to point out that some people's perceptions were based on errors, and see if that made them change the way they thing. Ross's using a computer that doesn't meet the minimum system requirements, for example, yet saying that iTunes is bloated because it's slow on his computer. Or the memory issue, which is a very common comment, which really doesn't make that much of a difference. I was thinking that the more technically inclined people here might be willing to consider that some of their reasons could be mitigated by, well, reason. I guess I was wrong. (The only person interested in the memory question, admittedly quite technical, was CarlJung; I was hoping someone could tell me if Windows manages memory in a similar manner.)

No big deal; I've gotten plenty of comments from others who are more open minded on other forums.

[HermanHiddema]

Kirkmc, you seem to have your mind long made up about this issue, and you have ready-made excuses on why the opinion of those who disagree with you is marginal and not as valid as yours.

Why are you still trolling for responses here then? Why not just go ahead and write the article the way you clearly want to anyways?

(The only person interested in the memory question, admittedly quite technical, was CarlJung; I was hoping someone could tell me if Windows manages memory in a similar manner.)

I started to write an explanation of the memory issue, but decided not to post it, because it was too technical and not really important.

In general, you seem to display a layman's understanding of memory management, and you should not, perhaps, use that limited understanding to argue for or against memory bloat issues. It sometimes reads a bit like the famous [sl=GetStrongAtKibitz]Get Strong At Kibitz[/sl] cover.

Windows and OSX (and other operating systems) manage memory in very similar ways. Swapping, for example, is not a choice by the program, but by the operating system. A program cannot "hold on" to physical memory. The OS decides which programs get to keep their stuff in RAM, and which get swapped to disk. A program can try to hold on to memory by using it, because the OS will generally swap out memory pages that are not used often, but that's no guarantee. And programs that use memory for the purpose of keeping it in RAM are frowned on, very much.

As to the bloat issue, I can say that when I was studying Computer Science, we were advised to write modules in such a way that they do one thing, and do it well. Don't tag on extra baggage. Don't write one module to do ten things, write ten modules that do one each. The same is often true for programs. Do one thing, do it well. The Mozilla project, for example, separated out the web browser and called it Firefox. Before that it was a web browser, email client, news reader, chat program, etc, etc, etc. Bloat. Firefox is much more efficient, it does one thing and does it well. But on the other hand, it was rather easy to have one program that allowed you to access everything on the internet in one go.

iTunes plays media, manages media, gives access to the store, synchronizes with your ipod, etc. If you don't have an ipod, then the ipod synch part is bloat. If you don't want to buy music online, then the store part is bloat. If you don't want to play music on your computer, then the music player is bloat. This is a choice that Apple made. They want thing to work out of the box. They want it so that if you buy music online, you don't have to go any extra steps to be able to play it, or put it on your iPod. You don't have to download a separate media player, you don't have to download a synch program. It just works. That doesn't mean it's not bloated, it just means there's a reason for the bloat: it's user friendly.

It's like buying a station wagon or SUV. You can buy a much smaller car and get around just as well. You can then rent a van when you need space to move stuff. But with the station wagon, you don't need to rent an extra van, it just works out of the box for transporting a lot of stuff. Yes, it is bloated, but that's a choice that you made. You sacrificed general efficiency in exchange for ease of use.

Personally, I prefer programs that do one thing well. But I'm computer savvy. I have no trouble at all finding and downloading applications that fit my needs. I have no objection to going trough some extra effort to get separate programs for every task. Most people just want things to work. The computer should be a magic box that just does what they want. That's why iTunes is the way it is.

[CarlJung]

Windows and OSX (and other operating systems) manage memory in very similar ways. Swapping, for example, is not a choice by the program, but by the operating system. A program cannot "hold on" to physical memory. The OS decides which programs get to keep their stuff in RAM, and which get swapped to disk. A program can try to hold on to memory by using it, because the OS will generally swap out memory pages that are not used often, but that's no guarantee. And programs that use memory for the purpose of keeping it in RAM are frowned on, very much.

I would guess that Windows has dynamic IO cache as well, but I'm not sure. Additionally it has something called SuperFetch that was introduced with Vista to make the computer snappier.

http://www.osnews.com/story/21471/Super ... orks_Myths

[flOvermind]

I was hoping someone could tell me if Windows manages memory in a similar manner.

No, AFAIK Windows doesn't, at least not exactly. Using Windows you'll always have plenty of free memory, which surely seems a waste.
On Linux it's very similar to OSX, all free memory is used up as cache, and can be given back at any time.

But I think you're misunderstanding what this feature is actually doing. I'll try to elaborate:

First there is the distinction between virtual memory and physical memory. Each application has 4 GB of virtual memory space (on a 32 bit system, more on a 64 bit system). The application can request as much of that virtual memory as it wants, up to the 4 GB limit (actually 3 GB, because the operating system needs a bit of it). It doesn't really matter how much the application requests, as this is just virtual memory.

There are several uses for virtual memory:
- The application just needs some anonymous memory. When it requests the memory, nothing actually happens. On first use, physical memory is allocated. This memory stays allocated until the application explicitly frees it. This is the memory that actually matters.
- The application needs a file (e.g. a library): The file is mapped in the virtual memory. When it is first accessed, physical memory is allocated and a part of the file is read into it. This memory has the property that it can be freed at any time, at the cost of some performance because the file has to be read again on next use. This physical memory can also be shared between multiple processes that need the same file. This is especially important because there are many libraries in the system that are needed by most programs.
(there are a few more, but these are the most important ones)

Now let's get to the physical memory. Applications never use physical memory directly, it is managed by the operating system. The operating system can basically do the following with physical memory:
- Satisfy the virtual memory needs of applications.
- Use it as backing store for mapped files.
- Cache data from other file system operations.

What is really interesting for the performance of a system is of course the physical memory. Virtual memory basically never runs out (at least with 64 bit systems).
What happens now when physical memory runs out? The operating system has several possibilities:
- Swap some anonymous memory.
- Throw away the memory of a mapped file. It may be necessary to write changes to disk first.
- Reduce the file system cache size.

There is actually no way for the operating system to just "throw away" some memory of an application, it always has to be swapped. But it can throw away the backing store of a mapped file, since it can just re-load it from disk at any time. And of course it can do with the file system cache whatever it wants.
What will the operating system actually do? It depends on the situation. One might think that swapping is always worse as long as you can reduce the cache size, but that's not necessarily true. If memory is used relatively infrequent, swapping might well be the better choice.

The article you linked above does talk about total memory consumption, and why it is always near 100%. That's simply because the operating system is always filling it up with the file system cache, and that's nothing to worry about.

Memory consumption of single processes is a different matter. Here you basically have two numbers: Allocated memory, and committed memory (sometimes incorrectly called virtual and physical). Allocated memory is irrelevant, since that's just memory that the application requested but did not actually used. Committed memory on the other hand is memory the application actually uses, and the operating system can't do anything about it without sacrificing some performance. The only memory the operating system can just throw away without affecting anyone is the cache, but you will never see the cache counted towards the memory consumption of a process.

So if an application uses 300 MB memory according to the task manager, it actually needs it. It may be that some of this memory is only used infrequently, and there is not much harm in swapping it, but it still means the application needs the memory, at least as far as the operating system is concerned.

This explanation applies exactly to Linux and OSX (and probably every other Unix out there). It also mostly applies to Windows. It works slight different, but the conclusions are still the same.

[kirkmc]

Windows and OSX (and other operating systems) manage memory in very similar ways. Swapping, for example, is not a choice by the program, but by the operating system. A program cannot "hold on" to physical memory. The OS decides which programs get to keep their stuff in RAM, and which get swapped to disk. A program can try to hold on to memory by using it, because the OS will generally swap out memory pages that are not used often, but that's no guarantee. And programs that use memory for the purpose of keeping it in RAM are frowned on, very much.

As to the bloat issue, I can say that when I was studying Computer Science, we were advised to write modules in such a way that they do one thing, and do it well. Don't tag on extra baggage. Don't write one module to do ten things, write ten modules that do one each. The same is often true for programs. Do one thing, do it well. The Mozilla project, for example, separated out the web browser and called it Firefox. Before that it was a web browser, email client, news reader, chat program, etc, etc, etc. Bloat. Firefox is much more efficient, it does one thing and does it well. But on the other hand, it was rather easy to have one program that allowed you to access everything on the internet in one go.

iTunes plays media, manages media, gives access to the store, synchronizes with your ipod, etc. If you don't have an ipod, then the ipod synch part is bloat. If you don't want to buy music online, then the store part is bloat. If you don't want to play music on your computer, then the music player is bloat. This is a choice that Apple made. They want thing to work out of the box. They want it so that if you buy music online, you don't have to go any extra steps to be able to play it, or put it on your iPod. You don't have to download a separate media player, you don't have to download a synch program. It just works. That doesn't mean it's not bloated, it just means there's a reason for the bloat: it's user friendly.

It's like buying a station wagon or SUV. You can buy a much smaller car and get around just as well. You can then rent a van when you need space to move stuff. But with the station wagon, you don't need to rent an extra van, it just works out of the box for transporting a lot of stuff. Yes, it is bloated, but that's a choice that you made. You sacrificed general efficiency in exchange for ease of use.

Personally, I prefer programs that do one thing well. But I'm computer savvy. I have no trouble at all finding and downloading applications that fit my needs. I have no objection to going trough some extra effort to get separate programs for every task. Most people just want things to work. The computer should be a magic box that just does what they want. That's why iTunes is the way it is.

But don't all modern programming tools allow such a modular approach within applications? (That's what other developers have told me.) The fact that features exist doesn't mean they're being used, or that they're using memory. They're called when needed, and therefore don't affect the program overall. It's not like a program needs to run all the code it has every time you click a button...

Regarding the memory thing, I'm not a developer, but I've discussed this with developers, and I certainly simplified. From what I understand, though, on OS X, a program will actually hold on to memory (or, to be more precise, store its code or data in memory pages) if it's using it, and after a certain time, it will free up that code or data allowing the OS to reclaim it. For this reason - and for years it's been like this - the standard advice given to people using Macs is that they basically never need to quit applications. As long as the app isn't doing anything, it'll get to a point when its memory can be reclaimed, and, in spite of what it may display as "real memory" that memory isn't being used. Naturally, swapping comes in to play here as well, but the memory figures that OS X displays are more theoretical than real.

[kirkmc]

So if an application uses 300 MB memory according to the task manager, it actually needs it. It may be that some of this memory is only used infrequently, and there is not much harm in swapping it, but it still means the application needs the memory, at least as far as the operating system is concerned.

This explanation applies exactly to Linux and OSX (and probably every other Unix out there). It also mostly applies to Windows. It works slight different, but the conclusions are still the same.

That's not what I've been told regarding OS X. I've been told that "inactive memory" is part of what a task manager (or top) will show for a program as "real memory" (what you call "committed memory"). Because of this, the OS is much more flexible about allocating that memory to other apps. I can see it sometimes in a tool I have that monitors memory; there are some operations that can reduce the total amount of memory used by my Mac by a huge amount. This is not just system memory (what's called "wired memory" on OS X), but overall memory usage by app.

Also, on OS X, if a computer has more RAM, apps will use more memory. I have two Macs, one with 2 GB RAM and the other with 4 GB. The latter uses easily twice as much RAM for the same apps as the former, though the apps are doing the same thing. I think on OS X - again, from what I've been told by developers - has a kind of "monte carlo" system, like winning by a half-point, when it deals with memory. Rather than worrying about details, it just grabs a lot, then doles it out more efficiently.

[flOvermind]

I've been told that "inactive memory" is part of what a task manager (or top) will show for a program as "real memory" (what you call "committed memory").

"Inactive memory" is very similar to the file system cache, and is a part of the total memory summary ("system memory"). It is not counted towards processes, and it can indeed simply be considered as "free".

See http://support.apple.com/kb/ht1342

This is not just system memory (what's called "wired memory" on OS X), but overall memory usage by app.

"Wired memory" is something entirely different. That's virtual memory that can not be swapped. It's mostly what the operating system itself (the kernel) uses. Applications usually should not request wired memory directly, but some do it anyway.

Also, on OS X, if a computer has more RAM, apps will use more memory. I have two Macs, one with 2 GB RAM and the other with 4 GB. The latter uses easily twice as much RAM for the same apps as the former, though the apps are doing the same thing.

Yes, that's what I explained in my previous post
It's not the apps that will use more memory, it's the buffers and cache of the operating system.

I think on OS X - again, from what I've been told by developers - has a kind of "monte carlo" system, like winning by a half-point, when it deals with memory. Rather than worrying about details, it just grabs a lot, then doles it out more efficiently.

Ok, that's a gross oversimplification. But to sum it up: Yes, the operating system does that. The applications do not.

[kirkmc]

(Sorry for not alternating quotes and texts...)

You say it's not that the apps use more memory, but the system shows that memory as being allocated to the apps. So what you say is more or less what I said, in a different way, that the figure shown as "real memory" is not a valid figure as to how much memory an app is using. Is that correct?

And the other point about the OS basically using what's available also can explain why, for one poster, iTunes is using 300 MB, or why, for me, it uses twice as much memory on one of my Macs as another.

My point being, as I tried to say above, that the amount of memory used by an app really means nothing. Though I'm not totally clear whether this number is closer to reality on Windows... From what you say above, this isn't the case; does that mean that any given app's memory figure is realistic, or is it higher or lower than the amount of memory that the app really _needs_?

[HermanHiddema]

But don't all modern programming tools allow such a modular approach within applications? (That's what other developers have told me.) The fact that features exist doesn't mean they're being used, or that they're using memory. They're called when needed, and therefore don't affect the program overall. It's not like a program needs to run all the code it has every time you click a button...

Yes and no

Yes, all modern programming languages allow a modular approach, and modern programs can load modules when they are needed. There's also approaches like JIT compiling (Just-In-Time compiling), which allow programs to be more efficient by postponing compilation until needed, when more information is available to allow optimization of the compilation. Certainly, programs today are a lot more flexible in this regard. But loading modules takes time, and for many scenarios, developers do not consider that desirable. So they preload certain modules to make the application more responsive.

One of the most common complaints against Mozilla, back in the day, was that it took so long to start it. Internet Explorer loaded much much faster. That was because windows preloaded most of the modules required for IE. So the start-up was a matter of a few seconds, rather than the dozens of seconds that Mozilla took. So of course, people complained to Mozilla that their browser was slow. And so Mozilla introduced a "preloading" config option, where you could make the system load most of the Mozilla modules in advance, and make it keep them in memory when the browser closed, so that it started faster. Of course, those preloaded modules use memory. Even if you never used IE, those preloaded IE modules were there.

And programs that use a monolithic approach (one application does many things) will often cross-link their parts, because that gives a more pleasant user experience. If one part of the program is aware of the options provided by other parts, then it can do smart things. Context sensitive options. Previews. You like this song? There's more by this artist at the store! You've played this song a lot on your iPod, I'll put it in your favorites list, etc. So a few of those modules are already needed for the nice stuff.

Again, you're trading off a pleasant user experience with a certain amount of bloat. But if you do not have an iPod, and never buy music online, then its bloat that provides no benefit. It's mostly unobtrusive, but not quite. It's mostly costless, but not quite. So the people that complain about bloat the most are those that use the fewest options of the program.

Regarding the memory thing, I'm not a developer, but I've discussed this with developers, and I certainly simplified. From what I understand, though, on OS X, a program will actually hold on to memory (or, to be more precise, store its code or data in memory pages) if it's using it, and after a certain time, it will free up that code or data allowing the OS to reclaim it. For this reason - and for years it's been like this - the standard advice given to people using Macs is that they basically never need to quit applications. As long as the app isn't doing anything, it'll get to a point when its memory can be reclaimed, and, in spite of what it may display as "real memory" that memory isn't being used. Naturally, swapping comes in to play here as well, but the memory figures that OS X displays are more theoretical than real.

flOvermind has written some excellent stuff on this issue, which covers most of this. Memory use is a tricky thing to measure. There's not just the IO cache, but also things like shared memory. The Chrome browser, for example, uses a separate process for each tab (and each plugin). Those processes share a lot of their memory. But a system tool may report each of them as using it. For an example of how this can mislead, see: http://www.ghacks.net/2009/06/21/web-br ... all-wrong/