Re: Network memory allocation failures

--zx4FCpZtqtKETZ7O
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Sep 07, 2010, Jeremy Chadwick wrote:
>=20
> I figured there might memory exhaustion of sorts, possibly in the bce(4)
> driver itself, that could cause the OP's problem.  bce(4) might not be
> the problem at all.  But the OP's issue seems to only occur when
> transmitting data, not receiving:
>=20
> http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058708.h=
tml

More information:

Looks like 100M wasn't enough of a test burst to tickle the problem in
my original message... 10G is, though.  It's definitely happening in
both directions.

Upgraded to -STABLE on one of the two machines last night, running
GENERIC.

FreeBSD obb 8.1-STABLE FreeBSD 8.1-STABLE #0: Tue Sep  7 19:48:55 PDT 2010 =
    root@obb:/usr/obj/usr/src/sys/GENERIC  amd64


Outgoing:

    obb# scp testfile root@holp:/usr/local/tmp/
    testfile                            8%  856MB  37.6MB/s   04:09 ETA
    Write failed: Cannot allocate memory
    lost connection
    obb# scp testfile root@holp:/usr/local/tmp/
    testfile                            0%   72MB  34.3MB/s   04:56 ETA
    Write failed: Cannot allocate memory
    lost connection

Incoming:

    obb# scp root@holp:/usr/local/tmp/testfile .
    testfile                            6%  670MB  31.9MB/s   04:59 ETA
    Write failed: Cannot allocate memory
    lost connection
    obb# scp root@holp:/usr/local/tmp/testfile .
    testfile                            1%  118MB  39.3MB/s   04:17 ETA
    Write failed: Cannot allocate memory
    lost connection
    obb# scp root@holp:/usr/local/tmp/testfile .
    testfile                           15% 1613MB  29.0MB/s   04:57 ETA
    Write failed: Cannot allocate memory
    lost connection



> The 2nd-to-last paragraph there is worth noting, specifically how
> limiting maximum addressable memory to 32GB via loader.conf seems to
> work around the issue.

I'd no longer consider this a coincidence, limiting the memory to 16G
eliminates the issue completely.  I'll retest with 32G today.

Incoming:

    obb# scp root@holp:/usr/local/tmp/testfile testfile2
    testfile                    100%   10GB  17.8MB/s   09:35
    obb# scp root@holp:/usr/local/tmp/testfile testfile2
    testfile                    100%   10GB  17.0MB/s   10:02

Outgoing:

    obb# scp testfile root@holp:/usr/local/tmp/testfile2
    testfile                    100%   10GB  35.7MB/s   04:47
    obb# scp testfile root@holp:/usr/local/tmp/testfile2
    testfile                    100%   10GB  35.4MB/s   04:49

=20
> There were other problems with the systems in question back in July, it
> seems.  I assume these got hammered out somehow:
>=20
> http://www.mail-archive.com/[e-mail]/msg111408.html

To a degree -- the initial install and cpu count problems are all fixed
up, thanks to help from the list.  The Intel 10G panics were stifled
with a newer driver from Intel's site, but I ran out of time to do
any serious testing with it, and just ended up using the broadcoms to
satisfy my time constraint.

--
Mahlon E. Smith =20
http://www.martini.nu/contact.html

--zx4FCpZtqtKETZ7O
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----

iD8DBQFMh58E1bsjBDapbeMRAg7qAJ4pGqjom+X+D0G06KzaLqR/d7gn5ACglvjL
LBf1URS7U4R1TcZ9O4PrBCE=
=Nus7
-----END PGP SIGNATURE-----

--zx4FCpZtqtKETZ7O--