<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	>
<channel>
	<title>Comments on: One myth less</title>
	<atom:link href="http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/</link>
	<description>a hacker analysis of life</description>
	<pubDate>Thu, 20 Nov 2008 10:48:18 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.6.2</generator>
		<item>
		<title>By: danilocesar</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21120</link>
		<dc:creator>danilocesar</dc:creator>
		<pubDate>Sat, 31 May 2008 22:02:10 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21120</guid>
		<description>Roger! =)</description>
		<content:encoded><![CDATA[<p>Roger! =)</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Gustavo Sverzut Barbieri</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21119</link>
		<dc:creator>Gustavo Sverzut Barbieri</dc:creator>
		<pubDate>Sat, 31 May 2008 19:53:15 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21119</guid>
		<description>Danilo,

What I tried to explain is that free(3) does not communicate directly to kernel, instead it will use an user space memory allocator, in UNIX case it's common to use LibC's. This user space memory allocator will in turn talk to kernel using sbrk(2) or mmap(2) syscalls, requesting both heap and anonymous memory changes.

Python adds yet more layers to this. Default C/Python will use a more customized memory allocator on top of LibC's, I believe that's also the case for MacOSX.

As for releasing memory when some object becomes unreferenced, you need to evaluate in which layer. Most of the times it will not give memory back directly to kernel, but will for sure give to the direct layer (in this case, Python's memory allocator), which can delay propagation of such "memory released" event, in the hope someone will need this memory soon, that's an optimization issue: every time you need to go to deeper layers, you pay a higher price.</description>
		<content:encoded><![CDATA[<p>Danilo,</p>
<p>What I tried to explain is that free(3) does not communicate directly to kernel, instead it will use an user space memory allocator, in UNIX case it&#8217;s common to use LibC&#8217;s. This user space memory allocator will in turn talk to kernel using sbrk(2) or mmap(2) syscalls, requesting both heap and anonymous memory changes.</p>
<p>Python adds yet more layers to this. Default C/Python will use a more customized memory allocator on top of LibC&#8217;s, I believe that&#8217;s also the case for MacOSX.</p>
<p>As for releasing memory when some object becomes unreferenced, you need to evaluate in which layer. Most of the times it will not give memory back directly to kernel, but will for sure give to the direct layer (in this case, Python&#8217;s memory allocator), which can delay propagation of such &#8220;memory released&#8221; event, in the hope someone will need this memory soon, that&#8217;s an optimization issue: every time you need to go to deeper layers, you pay a higher price.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: danilocesar</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21118</link>
		<dc:creator>danilocesar</dc:creator>
		<pubDate>Sat, 31 May 2008 19:04:29 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21118</guid>
		<description>Yes,but how libc release memory to kernel? Does Python use nmap to allocate memory? I don't know another way to return unused memory to kernel without close application...

Btw, in MacOSX, python doesn't release memory when it loses an object reference...</description>
		<content:encoded><![CDATA[<p>Yes,but how libc release memory to kernel? Does Python use nmap to allocate memory? I don&#8217;t know another way to return unused memory to kernel without close application&#8230;</p>
<p>Btw, in MacOSX, python doesn&#8217;t release memory when it loses an object reference&#8230;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Gustavo Sverzut Barbieri</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21117</link>
		<dc:creator>Gustavo Sverzut Barbieri</dc:creator>
		<pubDate>Sat, 31 May 2008 14:46:02 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21117</guid>
		<description>Danilo,

Python is something totally different.

This post was about POSIX libC free(3) implementation accepting NULL as parameter. Since other parts of libC does not accept, mostly with good reason to (ie: strdup(3)), I just assumed that free() also had that constraint, a myth to me.

If you pay attention to how both free(3) and strdup(3) are implemented, you can easily guess they work as expected. The former just uses the given address as an identifier for its own internal structures that maps given memory chunks (with malloc/realloc) to pages got from kernel VM, so the contents of that address is not touched. The latter, on the other hand, will use the contents of that pointer.

As for Danilo's comment: free(3) does not guarantee to give memory back to OS. LibC malloc/realloc/free will get pages from kernel's VM and then slice it as requested, distributing pieces to users of malloc/realloc. When you free some given memory, it will just mark the slice as unused. If the slice was the last used of a page, this page can be released to the underlying OS, but this might not be the case in some circumstances, due various optimizations/heuristics.

This is one of the reasons why you free() some memory and use it later and do not get a SEGV: to the kernel (VM), the page is still valid, so no signal is sent to the process.

Just for completeness: Python is a virtual machine and objects are automatically collected when they have no references left. Default C/Python uses reference-counting and 2-phase garbage collector, one quick and other more complete to remove garbage with circular-dependency. Also by default it uses another memory allocator (on top of libC malloc) to  speed up some common cases.  With that in mind, when you "del object" in Python, you're just releasing the reference to it. This reference can be the last, in this case object's memory would be returned to python memory allocator, that can return to libC that can return to kernel! :-)</description>
		<content:encoded><![CDATA[<p>Danilo,</p>
<p>Python is something totally different.</p>
<p>This post was about POSIX libC free(3) implementation accepting NULL as parameter. Since other parts of libC does not accept, mostly with good reason to (ie: strdup(3)), I just assumed that free() also had that constraint, a myth to me.</p>
<p>If you pay attention to how both free(3) and strdup(3) are implemented, you can easily guess they work as expected. The former just uses the given address as an identifier for its own internal structures that maps given memory chunks (with malloc/realloc) to pages got from kernel VM, so the contents of that address is not touched. The latter, on the other hand, will use the contents of that pointer.</p>
<p>As for Danilo&#8217;s comment: free(3) does not guarantee to give memory back to OS. LibC malloc/realloc/free will get pages from kernel&#8217;s VM and then slice it as requested, distributing pieces to users of malloc/realloc. When you free some given memory, it will just mark the slice as unused. If the slice was the last used of a page, this page can be released to the underlying OS, but this might not be the case in some circumstances, due various optimizations/heuristics.</p>
<p>This is one of the reasons why you free() some memory and use it later and do not get a SEGV: to the kernel (VM), the page is still valid, so no signal is sent to the process.</p>
<p>Just for completeness: Python is a virtual machine and objects are automatically collected when they have no references left. Default C/Python uses reference-counting and 2-phase garbage collector, one quick and other more complete to remove garbage with circular-dependency. Also by default it uses another memory allocator (on top of libC malloc) to  speed up some common cases.  With that in mind, when you &#8220;del object&#8221; in Python, you&#8217;re just releasing the reference to it. This reference can be the last, in this case object&#8217;s memory would be returned to python memory allocator, that can return to libC that can return to kernel! <img src='http://blog.gustavobarbieri.com.br/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: danilocesar</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21116</link>
		<dc:creator>danilocesar</dc:creator>
		<pubDate>Sat, 31 May 2008 06:39:31 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21116</guid>
		<description>I committed a similar mistake some days ago.

I was sure free()/del release memory, but don't turn it back to OS. 
But it's not true in python. del call returns allocated memory to OS.</description>
		<content:encoded><![CDATA[<p>I committed a similar mistake some days ago.</p>
<p>I was sure free()/del release memory, but don&#8217;t turn it back to OS.<br />
But it&#8217;s not true in python. del call returns allocated memory to OS.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Gustavo Sverzut Barbieri</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21112</link>
		<dc:creator>Gustavo Sverzut Barbieri</dc:creator>
		<pubDate>Tue, 20 May 2008 14:41:44 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21112</guid>
		<description>Although I plan to do no code in BREW, it's good to state so here.

To be clear to others: this is about POSIX. Not every system implements it.</description>
		<content:encoded><![CDATA[<p>Although I plan to do no code in BREW, it&#8217;s good to state so here.</p>
<p>To be clear to others: this is about POSIX. Not every system implements it.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: BenB</title>
		<link>http://blog.gustavobarbieri.com.br/2008/05/20/one-myth-less/#comment-21111</link>
		<dc:creator>BenB</dc:creator>
		<pubDate>Tue, 20 May 2008 14:04:37 +0000</pubDate>
		<guid isPermaLink="false">http://blog.gustavobarbieri.com.br/?p=96#comment-21111</guid>
		<description>It will crash on BREW (brew.qualcomm.com)  but that's just because BREW is different - you have to use FREE() in any case, and FREEIF() if you want the null pointer guard.</description>
		<content:encoded><![CDATA[<p>It will crash on BREW (brew.qualcomm.com)  but that&#8217;s just because BREW is different - you have to use FREE() in any case, and FREEIF() if you want the null pointer guard.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

<!-- Dynamic Page Served (once) in 0.372 seconds -->
