Failed to reserve contiguous memory

we have a cluster of 2 nodes running Windows 2000 SP4 with SQL Server
Enterprise Edition SP3.
There are 4 instances installed with 2 instances on each node.
We have started to have a problem over the last 2 weeks where jobs are
unable to start and return the error -
"Error 22022: Unable to post notification to SQLServerAgent (reason:
MapViewOfFile() returned error 8, Not enough storage is available to process
this"
looking at the error logs for both instances further errors are returned as
can be seen at the bottom of this post.
The instances that are returning the errors are both on the same node, to
investigate further their preferred nodes are switching over tonight, can you
give me some more insight into what type of errors we are experiencing.
INSTANCE ES
2006-03-08 09:06:45.62 spid68 Buffer Distribution: Stolen=892 Free=453
Procedures=6196
Inram=0 Dirty=2110 Kept=0
I/O=0, Latched=132, Other=159592
2006-03-08 09:06:45.62 spid68 Buffer Counts: Commited=169375
Target=216843 Hashed=161834
InternalReservation=140 ExternalReservation=0 Min Free=128
2006-03-08 09:06:45.62 spid68 Procedure Cache: TotalProcs=2204
TotalPages=6196 InUsePages=3584
2006-03-08 09:06:45.62 spid68 Dynamic Memory Manager: Stolen=7088 OS
Reserved=1728
OS Committed=1708
OS In Use=1694
Query Plan=6095 Optimizer=0
General=2078
Utilities=8 Connection=193
2006-03-08 09:06:45.63 spid68 Global Memory Objects: Resource=1737
Locks=347
SQLCache=140 Replication=2
LockBytes=2 ServerGlobal=44
Xact=18
2006-03-08 09:06:45.63 spid68 Query Memory Manager: Grants=0 Waiting=0
Maximum=159247 Available=159247
2006-03-08 09:06:48.29 spid68 Error: 17803, Severity: 20, State: 12
2006-03-08 09:06:48.29 spid68 Insufficient memory available..
2006-03-08 09:06:56.21 spid68 WARNING: Failed to reserve contiguous
memory of Size= 262144.
2006-03-08 09:06:56.23 spid68 Buffer Distribution: Stolen=924 Free=558
Procedures=6059
Inram=0 Dirty=2110 Kept=0
I/O=0, Latched=132, Other=159592
2006-03-08 09:06:56.23 spid68 Buffer Counts: Commited=169375
Target=216554 Hashed=161834
InternalReservation=272 ExternalReservation=0 Min Free=128
2006-03-08 09:06:56.23 spid68 Procedure Cache: TotalProcs=2199
TotalPages=6059 InUsePages=3528
2006-03-08 09:06:56.23 spid68 Dynamic Memory Manager: Stolen=6983 OS
Reserved=1728
OS Committed=1708
OS In Use=1685
Query Plan=5981 Optimizer=0
General=2078
Utilities=8 Connection=193
2006-03-08 09:06:56.23 spid68 Global Memory Objects: Resource=1737
Locks=347
SQLCache=140 Replication=2
LockBytes=2 ServerGlobal=44
Xact=18
2006-03-08 09:06:56.23 spid68 Query Memory Manager: Grants=0 Waiting=0
Maximum=159247 Available=159247
2006-03-08 09:06:56.23 spid68 Error: 17803, Severity: 20, State: 12
2006-03-08 09:06:56.23 spid68 Insufficient memory available..
INSTANCE FR
2006-03-07 09:34:16.15 spid78 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:34:16.18 spid78 Buffer Distribution: Stolen=1406 Free=5648
Procedures=8709
Inram=0 Dirty=10812 Kept=0
I/O=0, Latched=122, Other=238389
2006-03-07 09:34:16.18 spid78 Buffer Counts: Commited=265086
Target=276059 Hashed=249323
InternalReservation=180 ExternalReservation=0 Min Free=128
2006-03-07 09:34:16.18 spid78 Procedure Cache: TotalProcs=2430
TotalPages=8709 InUsePages=5151
2006-03-07 09:34:16.18 spid78 Dynamic Memory Manager: Stolen=10115 OS
Reserved=1392
OS Committed=1355
OS In Use=1342
Query Plan=8623 Optimizer=0
General=1822
Utilities=9 Connection=862
2006-03-07 09:34:16.20 spid78 Global Memory Objects: Resource=1493
Locks=56
SQLCache=142 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:34:16.20 spid78 Query Memory Manager: Grants=0 Waiting=0
Maximum=202611 Available=202611
2006-03-07 09:34:16.37 spid78 Error: 17803, Severity: 20, State: 12
2006-03-07 09:34:16.37 spid78 Insufficient memory available..
2006-03-07 09:36:09.73 spid78 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:36:09.74 spid78 Buffer Distribution: Stolen=1714 Free=5825
Procedures=8285
Inram=0 Dirty=10813 Kept=0
I/O=0, Latched=122, Other=238404
2006-03-07 09:36:09.74 spid78 Buffer Counts: Commited=265163
Target=286067 Hashed=249339
InternalReservation=286 ExternalReservation=0 Min Free=128
2006-03-07 09:36:09.74 spid78 Procedure Cache: TotalProcs=2424
TotalPages=8285 InUsePages=4827
2006-03-07 09:36:09.74 spid78 Dynamic Memory Manager: Stolen=9999 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8496 Optimizer=0
General=1822
Utilities=9 Connection=853
2006-03-07 09:36:09.74 spid78 Global Memory Objects: Resource=1493
Locks=56
SQLCache=142 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:36:09.74 spid78 Query Memory Manager: Grants=0 Waiting=0
Maximum=209646 Available=209646
2006-03-07 09:36:09.74 spid78 Error: 17803, Severity: 20, State: 12
2006-03-07 09:36:09.74 spid78 Insufficient memory available..
2006-03-07 09:40:44.73 spid90 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:40:44.75 spid90 Buffer Distribution: Stolen=1387 Free=6121
Procedures=8391
Inram=0 Dirty=10573 Kept=0
I/O=0, Latched=129, Other=238697
2006-03-07 09:40:44.75 spid90 Buffer Counts: Commited=265298
Target=291738 Hashed=249399
InternalReservation=304 ExternalReservation=112 Min Free=128
2006-03-07 09:40:44.75 spid90 Procedure Cache: TotalProcs=2482
TotalPages=8391 InUsePages=4895
2006-03-07 09:40:44.75 spid90 Dynamic Memory Manager: Stolen=9757 OS
Reserved=1392
OS Committed=1355
OS In Use=1336
Query Plan=8623 Optimizer=35
General=1830
Utilities=9 Connection=307
2006-03-07 09:40:44.75 spid90 Global Memory Objects: Resource=1493
Locks=292
SQLCache=145 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:40:44.75 spid90 Query Memory Manager: Grants=1 Waiting=0
Maximum=213526 Available=213393
2006-03-07 09:40:44.77 spid90 Error: 17803, Severity: 20, State: 12
2006-03-07 09:40:44.77 spid90 Insufficient memory available..
2006-03-07 09:40:48.69 spid90 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:40:48.69 spid90 Buffer Distribution: Stolen=1317 Free=6265
Procedures=8397
Inram=0 Dirty=10489 Kept=0
I/O=0, Latched=129, Other=238701
2006-03-07 09:40:48.69 spid90 Buffer Counts: Commited=265298
Target=290968 Hashed=249319
InternalReservation=311 ExternalReservation=40 Min Free=128
2006-03-07 09:40:48.69 spid90 Procedure Cache: TotalProcs=2483
TotalPages=8397 InUsePages=4882
2006-03-07 09:40:48.69 spid90 Dynamic Memory Manager: Stolen=9696 OS
Reserved=1392
OS Committed=1355
OS In Use=1338
Query Plan=8614 Optimizer=0
General=1828
Utilities=9 Connection=301
2006-03-07 09:40:48.69 spid90 Global Memory Objects: Resource=1493
Locks=249
SQLCache=145 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:40:48.69 spid90 Query Memory Manager: Grants=1 Waiting=0
Maximum=213975 Available=213917
2006-03-07 09:40:48.70 spid90 Error: 17803, Severity: 20, State: 12
2006-03-07 09:40:48.70 spid90 Insufficient memory available..
2006-03-07 09:41:02.89 spid85 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:41:02.91 spid85 Buffer Distribution: Stolen=1289 Free=6292
Procedures=8398
Inram=0 Dirty=10496 Kept=0
I/O=0, Latched=128, Other=238695
2006-03-07 09:41:02.91 spid85 Buffer Counts: Commited=265298
Target=291100 Hashed=249319
InternalReservation=311 ExternalReservation=0 Min Free=128
2006-03-07 09:41:02.91 spid85 Procedure Cache: TotalProcs=2484
TotalPages=8398 InUsePages=4882
2006-03-07 09:41:02.91 spid85 Dynamic Memory Manager: Stolen=9687 OS
Reserved=1392
OS Committed=1355
OS In Use=1334
Query Plan=8612 Optimizer=0
General=1822
Utilities=9 Connection=296
2006-03-07 09:41:02.91 spid85 Global Memory Objects: Resource=1493
Locks=249
SQLCache=145 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:41:02.91 spid85 Query Memory Manager: Grants=0 Waiting=0
Maximum=213745 Available=213745
2006-03-07 09:41:02.91 spid85 Error: 17803, Severity: 20, State: 12
2006-03-07 09:41:02.91 spid85 Insufficient memory available..
2006-03-07 09:42:12.80 spid83 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:42:12.80 spid83 Buffer Distribution: Stolen=1273 Free=6015
Procedures=8407
Inram=0 Dirty=10506 Kept=0
I/O=0, Latched=126, Other=238974
2006-03-07 09:42:12.80 spid83 Buffer Counts: Commited=265301
Target=286566 Hashed=249606
InternalReservation=306 ExternalReservation=0 Min Free=128
2006-03-07 09:42:12.80 spid83 Procedure Cache: TotalProcs=2489
TotalPages=8407 InUsePages=4892
2006-03-07 09:42:12.80 spid83 Dynamic Memory Manager: Stolen=9680 OS
Reserved=1392
OS Committed=1355
OS In Use=1334
Query Plan=8627 Optimizer=0
General=1816
Utilities=9 Connection=280
2006-03-07 09:42:12.80 spid83 Global Memory Objects: Resource=1493
Locks=214
SQLCache=145 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:42:12.80 spid83 Query Memory Manager: Grants=0 Waiting=0
Maximum=210349 Available=210349
2006-03-07 09:42:12.81 spid83 Error: 17803, Severity: 20, State: 12
2006-03-07 09:42:12.81 spid83 Insufficient memory available..
2006-03-07 09:43:01.86 spid83 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:43:01.88 spid83 Buffer Distribution: Stolen=1267 Free=6013
Procedures=8415
Inram=0 Dirty=10511 Kept=0
I/O=0, Latched=126, Other=238969
2006-03-07 09:43:01.88 spid83 Buffer Counts: Commited=265301
Target=283566 Hashed=249606
InternalReservation=304 ExternalReservation=0 Min Free=128
2006-03-07 09:43:01.88 spid83 Procedure Cache: TotalProcs=2490
TotalPages=8415 InUsePages=4895
2006-03-07 09:43:01.88 spid83 Dynamic Memory Manager: Stolen=9682 OS
Reserved=1392
OS Committed=1355
OS In Use=1334
Query Plan=8635 Optimizer=0
General=1816
Utilities=9 Connection=280
2006-03-07 09:43:01.88 spid83 Global Memory Objects: Resource=1493
Locks=186
SQLCache=145 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:43:01.88 spid83 Query Memory Manager: Grants=0 Waiting=0
Maximum=208224 Available=208224
2006-03-07 09:43:01.88 spid83 Error: 17803, Severity: 20, State: 12
2006-03-07 09:43:01.88 spid83 Insufficient memory available..
2006-03-07 09:44:51.07 spid83 WARNING: Failed to reserve contiguous
memory of Size= 393216.
2006-03-07 09:44:51.07 spid83 Buffer Distribution: Stolen=1246 Free=5938
Procedures=8451
Inram=0 Dirty=10539 Kept=0
I/O=0, Latched=125, Other=239070
2006-03-07 09:44:51.07 spid83 Buffer Counts: Commited=265369
Target=276731 Hashed=249734
InternalReservation=304 ExternalReservation=0 Min Free=128
2006-03-07 09:44:51.07 spid83 Procedure Cache: TotalProcs=2519
TotalPages=8451 InUsePages=4915
2006-03-07 09:44:51.07 spid83 Dynamic Memory Manager: Stolen=9697 OS
Reserved=1392
OS Committed=1355
OS In Use=1334
Query Plan=8669 Optimizer=0
General=1819
Utilities=9 Connection=284
2006-03-07 09:44:51.07 spid83 Global Memory Objects: Resource=1493
Locks=145
SQLCache=146 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:44:51.07 spid83 Query Memory Manager: Grants=0 Waiting=0
Maximum=203051 Available=203051
2006-03-07 09:44:51.08 spid83 Error: 17803, Severity: 20, State: 12
2006-03-07 09:44:51.08 spid83 Insufficient memory available..
2006-03-07 09:48:35.73 spid53 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:48:35.81 spid53 Buffer Distribution: Stolen=1852 Free=254
Procedures=8279
Inram=0 Dirty=8797 Kept=0
I/O=0, Latched=123, Other=275965
2006-03-07 09:48:35.81 spid53 Buffer Counts: Commited=295270
Target=295270 Hashed=284885
InternalReservation=301 ExternalReservation=60 Min Free=280
2006-03-07 09:48:35.81 spid53 Procedure Cache: TotalProcs=2532
TotalPages=8279 InUsePages=4576
2006-03-07 09:48:35.81 spid53 Dynamic Memory Manager: Stolen=10113 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8507 Optimizer=0
General=1824
Utilities=9 Connection=873
2006-03-07 09:48:35.82 spid53 Global Memory Objects: Resource=1493
Locks=98
SQLCache=146 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:48:35.82 spid53 Query Memory Manager: Grants=1 Waiting=0
Maximum=216625 Available=216547
2006-03-07 09:48:36.17 spid53 Error: 17803, Severity: 20, State: 12
2006-03-07 09:48:36.17 spid53 Insufficient memory available..
2006-03-07 09:52:20.51 spid86 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:52:20.53 spid86 Buffer Distribution: Stolen=1834 Free=0
Procedures=8449
Inram=0 Dirty=8873 Kept=0
I/O=0, Latched=126, Other=281582
2006-03-07 09:52:20.53 spid86 Buffer Counts: Commited=300864
Target=303340 Hashed=290581
InternalReservation=299 ExternalReservation=60 Min Free=512
2006-03-07 09:52:20.53 spid86 Procedure Cache: TotalProcs=2565
TotalPages=8449 InUsePages=4679
2006-03-07 09:52:20.53 spid86 Dynamic Memory Manager: Stolen=10265 OS
Reserved=1376
OS Committed=1339
OS In Use=1324
Query Plan=8677 Optimizer=0
General=1825
Utilities=9 Connection=867
2006-03-07 09:52:20.53 spid86 Global Memory Objects: Resource=1493
Locks=79
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:52:20.53 spid86 Query Memory Manager: Grants=1 Waiting=0
Maximum=223068 Available=222990
2006-03-07 09:52:20.53 spid86 Error: 17803, Severity: 20, State: 12
2006-03-07 09:52:20.53 spid86 Insufficient memory available..
2006-03-07 09:52:29.40 spid86 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:52:29.42 spid86 Buffer Distribution: Stolen=1844 Free=230
Procedures=8435
Inram=0 Dirty=8873 Kept=0
I/O=0, Latched=127, Other=281582
2006-03-07 09:52:29.42 spid86 Buffer Counts: Commited=301091
Target=303153 Hashed=290582
InternalReservation=302 ExternalReservation=60 Min Free=496
2006-03-07 09:52:29.42 spid86 Procedure Cache: TotalProcs=2563
TotalPages=8435 InUsePages=4666
2006-03-07 09:52:29.42 spid86 Dynamic Memory Manager: Stolen=10261 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8666 Optimizer=0
General=1825
Utilities=9 Connection=872
2006-03-07 09:52:29.42 spid86 Global Memory Objects: Resource=1493
Locks=74
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:52:29.42 spid86 Query Memory Manager: Grants=1 Waiting=0
Maximum=223086 Available=223008
2006-03-07 09:52:29.42 spid86 Error: 17803, Severity: 20, State: 12
2006-03-07 09:52:29.42 spid86 Insufficient memory available..
2006-03-07 09:53:43.58 spid76 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:53:43.59 spid76 Buffer Distribution: Stolen=1802
Free=24279 Procedures=8464
Inram=0 Dirty=8962 Kept=0
I/O=0, Latched=126, Other=255435
2006-03-07 09:53:43.59 spid76 Buffer Counts: Commited=299068
Target=299068 Hashed=264523
InternalReservation=303 ExternalReservation=0 Min Free=360
2006-03-07 09:53:43.59 spid76 Procedure Cache: TotalProcs=2580
TotalPages=8464 InUsePages=4689
2006-03-07 09:53:43.59 spid76 Dynamic Memory Manager: Stolen=10266 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8678 Optimizer=0
General=1824
Utilities=9 Connection=866
2006-03-07 09:53:43.59 spid76 Global Memory Objects: Resource=1493
Locks=71
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:53:43.59 spid76 Query Memory Manager: Grants=0 Waiting=0
Maximum=219938 Available=219938
2006-03-07 09:53:43.59 spid76 Error: 17803, Severity: 20, State: 12
2006-03-07 09:53:43.59 spid76 Insufficient memory available..
2006-03-07 09:54:17.44 spid85 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:54:17.44 spid85 Buffer Distribution: Stolen=1806
Free=23999 Procedures=8470
Inram=0 Dirty=8971 Kept=0
I/O=0, Latched=119, Other=255568
2006-03-07 09:54:17.44 spid85 Buffer Counts: Commited=298933
Target=298933 Hashed=264658
InternalReservation=302 ExternalReservation=0 Min Free=224
2006-03-07 09:54:17.44 spid85 Procedure Cache: TotalProcs=2580
TotalPages=8470 InUsePages=4696
2006-03-07 09:54:17.44 spid85 Dynamic Memory Manager: Stolen=10276 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8684 Optimizer=0
General=1824
Utilities=9 Connection=870
2006-03-07 09:54:17.44 spid85 Global Memory Objects: Resource=1493
Locks=70
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:54:17.44 spid85 Query Memory Manager: Grants=0 Waiting=0
Maximum=219821 Available=219821
2006-03-07 09:54:17.45 spid85 Error: 17803, Severity: 20, State: 12
2006-03-07 09:54:17.45 spid85 Insufficient memory available..
2006-03-07 09:55:01.91 spid85 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:55:01.92 spid85 Buffer Distribution: Stolen=1814
Free=23146 Procedures=8477
Inram=0 Dirty=8982 Kept=0
I/O=0, Latched=118, Other=256321
2006-03-07 09:55:01.92 spid85 Buffer Counts: Commited=298858
Target=298858 Hashed=265421
InternalReservation=295 ExternalReservation=0 Min Free=128
2006-03-07 09:55:01.92 spid85 Procedure Cache: TotalProcs=2584
TotalPages=8477 InUsePages=4709
2006-03-07 09:55:01.92 spid85 Dynamic Memory Manager: Stolen=10291 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8699 Optimizer=0
General=1825
Utilities=9 Connection=869
2006-03-07 09:55:01.92 spid85 Global Memory Objects: Resource=1494
Locks=70
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:55:01.92 spid85 Query Memory Manager: Grants=0 Waiting=0
Maximum=219759 Available=219759
2006-03-07 09:55:01.92 spid85 Error: 17803, Severity: 20, State: 12
2006-03-07 09:55:01.92 spid85 Insufficient memory available..
2006-03-07 09:56:15.35 spid87 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 09:56:15.36 spid87 Buffer Distribution: Stolen=1847
Free=23165 Procedures=8421
Inram=0 Dirty=8991 Kept=0
I/O=0, Latched=116, Other=256318
2006-03-07 09:56:15.36 spid87 Buffer Counts: Commited=298858
Target=314884 Hashed=265425
InternalReservation=294 ExternalReservation=60 Min Free=128
2006-03-07 09:56:15.36 spid87 Procedure Cache: TotalProcs=2588
TotalPages=8421 InUsePages=4621
2006-03-07 09:56:15.36 spid87 Dynamic Memory Manager: Stolen=10250 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8649 Optimizer=0
General=1825
Utilities=9 Connection=878
2006-03-07 09:56:15.36 spid87 Global Memory Objects: Resource=1493
Locks=70
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:56:15.36 spid87 Query Memory Manager: Grants=1 Waiting=0
Maximum=231843 Available=231765
2006-03-07 09:56:15.36 spid87 Error: 17803, Severity: 20, State: 12
2006-03-07 09:56:15.36 spid87 Insufficient memory available..
2006-03-07 09:56:39.77 spid87 WARNING: Failed to reserve contiguous
memory of Size= 2752512.
2006-03-07 09:56:39.77 spid87 Buffer Distribution: Stolen=1820
Free=22535 Procedures=8420
Inram=0 Dirty=8991 Kept=0
I/O=0, Latched=117, Other=256975
2006-03-07 09:56:39.77 spid87 Buffer Counts: Commited=298858
Target=314520 Hashed=266083
InternalReservation=292 ExternalReservation=60 Min Free=128
2006-03-07 09:56:39.77 spid87 Procedure Cache: TotalProcs=2587
TotalPages=8420 InUsePages=4621
2006-03-07 09:56:39.77 spid87 Dynamic Memory Manager: Stolen=10222 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8650 Optimizer=0
General=1825
Utilities=9 Connection=849
2006-03-07 09:56:39.77 spid87 Global Memory Objects: Resource=1493
Locks=70
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=29
2006-03-07 09:56:39.77 spid87 Query Memory Manager: Grants=1 Waiting=0
Maximum=231587 Available=231509
2006-03-07 09:56:39.78 spid87 Error: 17803, Severity: 20, State: 12
2006-03-07 09:56:39.78 spid87 Insufficient memory available..
2006-03-07 10:03:11.82 spid76 WARNING: Failed to reserve contiguous
memory of Size= 2752512.
2006-03-07 10:03:11.84 spid76 Buffer Distribution: Stolen=1791
Free=22216 Procedures=8476
Inram=0 Dirty=8763 Kept=0
I/O=0, Latched=125, Other=257487
2006-03-07 10:03:11.84 spid76 Buffer Counts: Commited=298858
Target=315219 Hashed=266375
InternalReservation=311 ExternalReservation=0 Min Free=128
2006-03-07 10:03:11.84 spid76 Procedure Cache: TotalProcs=2616
TotalPages=8476 InUsePages=4646
2006-03-07 10:03:11.84 spid76 Dynamic Memory Manager: Stolen=10267 OS
Reserved=1376
OS Committed=1339
OS In Use=1322
Query Plan=8690 Optimizer=0
General=1826
Utilities=9 Connection=854
2006-03-07 10:03:11.84 spid76 Global Memory Objects: Resource=1493
Locks=81
SQLCache=147 Replication=2
LockBytes=2 ServerGlobal=43
Xact=31
2006-03-07 10:03:11.84 spid76 Query Memory Manager: Grants=0 Waiting=0
Maximum=232111 Available=232111
2006-03-07 10:03:11.84 spid76 Error: 17803, Severity: 20, State: 12
2006-03-07 10:03:11.84 spid76 Insufficient memory available..
2006-03-07 10:43:21.28 spid78 DBCC CHECKDB (msdb) executed by
SPICERS_WH\CaseltonC found 0 errors and repaired 0 errors.
2006-03-07 10:46:04.68 spid88 WARNING: Failed to reserve contiguous
memory of Size= 2883584.
2006-03-07 10:46:04.69 spid88 Buffer Distribution: Stolen=2640
Free=10042 Procedures=9219
Inram=0 Dirty=9834 Kept=0
I/O=0, Latched=142, Other=291391
2006-03-07 10:46:04.69 spid88 Buffer Counts: Commited=323268
Target=338520 Hashed=301367
InternalReservation=309 ExternalReservation=5060 Min Free=128
2006-03-07 10:46:04.69 spid88 Procedure Cache: TotalProcs=2842
TotalPages=9219 InUsePages=5071
2006-03-07 10:46:04.69 spid88 Dynamic Memory Manager: Stolen=11856 OS
Reserved=1392
OS Committed=1355
OS In Use=1332
Query Plan=9591 Optimizer=0
General=1985
Utilities=519 Connection=903
2006-03-07 10:46:04.69 spid88 Global Memory Objects: Resource=1505
Locks=50
SQLCache=154 Replication=2
LockBytes=2 ServerGlobal=46
Xact=31
2006-03-07 10:46:04.69 spid88 Query Memory Manager: Grants=2 Waiting=0
Maximum=248670 Available=243607
2006-03-07 10:46:04.71 spid88 Error: 17803, Severity: 20, State: 12
2006-03-07 10:46:04.71 spid88 Insufficient memory available..
Please could I have a response to this post - it has now been over 3 business
days.
I'm looking for help with where I should start my investigation - I don't
understand the reason for the 'not enough storage space' message, as there is
plenty of storage space! The error logs suggest a problem with memory -
'Failed to reserve contiguous memory of Size= 262144'. Presumably this is the
underlying problem? If so, it must be due to something that has changed on
all the servers, as we have now seen the problem on 2 of the 4 instances (1
from each cluster). Should I be looking at the jobs that are running with a
view to identifying memory leaks, or is it more likely to be due to something
that has changed at OS level - perhaps a patch that has been applied?
Finally, does SQL Server SP4 contain anything to specifically fix this
problem?
At the moment, we're getting round the issue by stopping and restarting
instances every night, which is obviously far from satisfactory.
Regards,
David.
"David" wrote:

> we have a cluster of 2 nodes running Windows 2000 SP4 with SQL Server
> Enterprise Edition SP3.
> There are 4 instances installed with 2 instances on each node.
> We have started to have a problem over the last 2 weeks where jobs are
> unable to start and return the error -
> "Error 22022: Unable to post notification to SQLServerAgent (reason:
> MapViewOfFile() returned error 8, Not enough storage is available to process
> this"
> looking at the error logs for both instances further errors are returned as
> can be seen at the bottom of this post.
> The instances that are returning the errors are both on the same node, to
> investigate further their preferred nodes are switching over tonight, can you
> give me some more insight into what type of errors we are experiencing.
> INSTANCE ES
> 2006-03-08 09:06:45.62 spid68 Buffer Distribution: Stolen=892 Free=453
> Procedures=6196
> Inram=0 Dirty=2110 Kept=0
> I/O=0, Latched=132, Other=159592
> 2006-03-08 09:06:45.62 spid68 Buffer Counts: Commited=169375
> Target=216843 Hashed=161834
> InternalReservation=140 ExternalReservation=0 Min Free=128
> 2006-03-08 09:06:45.62 spid68 Procedure Cache: TotalProcs=2204
> TotalPages=6196 InUsePages=3584
> 2006-03-08 09:06:45.62 spid68 Dynamic Memory Manager: Stolen=7088 OS
> Reserved=1728
> OS Committed=1708
> OS In Use=1694
> Query Plan=6095 Optimizer=0
> General=2078
> Utilities=8 Connection=193
> 2006-03-08 09:06:45.63 spid68 Global Memory Objects: Resource=1737
> Locks=347
> SQLCache=140 Replication=2
> LockBytes=2 ServerGlobal=44
> Xact=18
> 2006-03-08 09:06:45.63 spid68 Query Memory Manager: Grants=0 Waiting=0
> Maximum=159247 Available=159247
> 2006-03-08 09:06:48.29 spid68 Error: 17803, Severity: 20, State: 12
> 2006-03-08 09:06:48.29 spid68 Insufficient memory available..
> 2006-03-08 09:06:56.21 spid68 WARNING: Failed to reserve contiguous
> memory of Size= 262144.
> 2006-03-08 09:06:56.23 spid68 Buffer Distribution: Stolen=924 Free=558
> Procedures=6059
> Inram=0 Dirty=2110 Kept=0
> I/O=0, Latched=132, Other=159592
> 2006-03-08 09:06:56.23 spid68 Buffer Counts: Commited=169375
> Target=216554 Hashed=161834
> InternalReservation=272 ExternalReservation=0 Min Free=128
> 2006-03-08 09:06:56.23 spid68 Procedure Cache: TotalProcs=2199
> TotalPages=6059 InUsePages=3528
> 2006-03-08 09:06:56.23 spid68 Dynamic Memory Manager: Stolen=6983 OS
> Reserved=1728
> OS Committed=1708
> OS In Use=1685
> Query Plan=5981 Optimizer=0
> General=2078
> Utilities=8 Connection=193
> 2006-03-08 09:06:56.23 spid68 Global Memory Objects: Resource=1737
> Locks=347
> SQLCache=140 Replication=2
> LockBytes=2 ServerGlobal=44
> Xact=18
> 2006-03-08 09:06:56.23 spid68 Query Memory Manager: Grants=0 Waiting=0
> Maximum=159247 Available=159247
> 2006-03-08 09:06:56.23 spid68 Error: 17803, Severity: 20, State: 12
> 2006-03-08 09:06:56.23 spid68 Insufficient memory available..
> INSTANCE FR
> 2006-03-07 09:34:16.15 spid78 WARNING: Failed to reserve contiguous
> memory of Size= 2883584.
> 2006-03-07 09:34:16.18 spid78 Buffer Distribution: Stolen=1406 Free=5648
> Procedures=8709
> Inram=0 Dirty=10812 Kept=0
> I/O=0, Latched=122, Other=238389
> 2006-03-07 09:34:16.18 spid78 Buffer Counts: Commited=265086
> Target=276059 Hashed=249323
> InternalReservation=180 ExternalReservation=0 Min Free=128
> 2006-03-07 09:34:16.18 spid78 Procedure Cache: TotalProcs=2430
> TotalPages=8709 InUsePages=5151
> 2006-03-07 09:34:16.18 spid78 Dynamic Memory Manager: Stolen=10115 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1342
> Query Plan=8623 Optimizer=0
> General=1822
> Utilities=9 Connection=862
> 2006-03-07 09:34:16.20 spid78 Global Memory Objects: Resource=1493
> Locks=56
> SQLCache=142 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:34:16.20 spid78 Query Memory Manager: Grants=0 Waiting=0
> Maximum=202611 Available=202611
> 2006-03-07 09:34:16.37 spid78 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:34:16.37 spid78 Insufficient memory available..
> 2006-03-07 09:36:09.73 spid78 WARNING: Failed to reserve contiguous
> memory of Size= 2883584.
> 2006-03-07 09:36:09.74 spid78 Buffer Distribution: Stolen=1714 Free=5825
> Procedures=8285
> Inram=0 Dirty=10813 Kept=0
> I/O=0, Latched=122, Other=238404
> 2006-03-07 09:36:09.74 spid78 Buffer Counts: Commited=265163
> Target=286067 Hashed=249339
> InternalReservation=286 ExternalReservation=0 Min Free=128
> 2006-03-07 09:36:09.74 spid78 Procedure Cache: TotalProcs=2424
> TotalPages=8285 InUsePages=4827
> 2006-03-07 09:36:09.74 spid78 Dynamic Memory Manager: Stolen=9999 OS
> Reserved=1376
> OS Committed=1339
> OS In Use=1322
> Query Plan=8496 Optimizer=0
> General=1822
> Utilities=9 Connection=853
> 2006-03-07 09:36:09.74 spid78 Global Memory Objects: Resource=1493
> Locks=56
> SQLCache=142 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:36:09.74 spid78 Query Memory Manager: Grants=0 Waiting=0
> Maximum=209646 Available=209646
> 2006-03-07 09:36:09.74 spid78 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:36:09.74 spid78 Insufficient memory available..
> 2006-03-07 09:40:44.73 spid90 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:40:44.75 spid90 Buffer Distribution: Stolen=1387 Free=6121
> Procedures=8391
> Inram=0 Dirty=10573 Kept=0
> I/O=0, Latched=129, Other=238697
> 2006-03-07 09:40:44.75 spid90 Buffer Counts: Commited=265298
> Target=291738 Hashed=249399
> InternalReservation=304 ExternalReservation=112 Min Free=128
> 2006-03-07 09:40:44.75 spid90 Procedure Cache: TotalProcs=2482
> TotalPages=8391 InUsePages=4895
> 2006-03-07 09:40:44.75 spid90 Dynamic Memory Manager: Stolen=9757 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1336
> Query Plan=8623 Optimizer=35
> General=1830
> Utilities=9 Connection=307
> 2006-03-07 09:40:44.75 spid90 Global Memory Objects: Resource=1493
> Locks=292
> SQLCache=145 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:40:44.75 spid90 Query Memory Manager: Grants=1 Waiting=0
> Maximum=213526 Available=213393
> 2006-03-07 09:40:44.77 spid90 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:40:44.77 spid90 Insufficient memory available..
> 2006-03-07 09:40:48.69 spid90 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:40:48.69 spid90 Buffer Distribution: Stolen=1317 Free=6265
> Procedures=8397
> Inram=0 Dirty=10489 Kept=0
> I/O=0, Latched=129, Other=238701
> 2006-03-07 09:40:48.69 spid90 Buffer Counts: Commited=265298
> Target=290968 Hashed=249319
> InternalReservation=311 ExternalReservation=40 Min Free=128
> 2006-03-07 09:40:48.69 spid90 Procedure Cache: TotalProcs=2483
> TotalPages=8397 InUsePages=4882
> 2006-03-07 09:40:48.69 spid90 Dynamic Memory Manager: Stolen=9696 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1338
> Query Plan=8614 Optimizer=0
> General=1828
> Utilities=9 Connection=301
> 2006-03-07 09:40:48.69 spid90 Global Memory Objects: Resource=1493
> Locks=249
> SQLCache=145 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:40:48.69 spid90 Query Memory Manager: Grants=1 Waiting=0
> Maximum=213975 Available=213917
> 2006-03-07 09:40:48.70 spid90 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:40:48.70 spid90 Insufficient memory available..
> 2006-03-07 09:41:02.89 spid85 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:41:02.91 spid85 Buffer Distribution: Stolen=1289 Free=6292
> Procedures=8398
> Inram=0 Dirty=10496 Kept=0
> I/O=0, Latched=128, Other=238695
> 2006-03-07 09:41:02.91 spid85 Buffer Counts: Commited=265298
> Target=291100 Hashed=249319
> InternalReservation=311 ExternalReservation=0 Min Free=128
> 2006-03-07 09:41:02.91 spid85 Procedure Cache: TotalProcs=2484
> TotalPages=8398 InUsePages=4882
> 2006-03-07 09:41:02.91 spid85 Dynamic Memory Manager: Stolen=9687 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1334
> Query Plan=8612 Optimizer=0
> General=1822
> Utilities=9 Connection=296
> 2006-03-07 09:41:02.91 spid85 Global Memory Objects: Resource=1493
> Locks=249
> SQLCache=145 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:41:02.91 spid85 Query Memory Manager: Grants=0 Waiting=0
> Maximum=213745 Available=213745
> 2006-03-07 09:41:02.91 spid85 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:41:02.91 spid85 Insufficient memory available..
> 2006-03-07 09:42:12.80 spid83 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:42:12.80 spid83 Buffer Distribution: Stolen=1273 Free=6015
> Procedures=8407
> Inram=0 Dirty=10506 Kept=0
> I/O=0, Latched=126, Other=238974
> 2006-03-07 09:42:12.80 spid83 Buffer Counts: Commited=265301
> Target=286566 Hashed=249606
> InternalReservation=306 ExternalReservation=0 Min Free=128
> 2006-03-07 09:42:12.80 spid83 Procedure Cache: TotalProcs=2489
> TotalPages=8407 InUsePages=4892
> 2006-03-07 09:42:12.80 spid83 Dynamic Memory Manager: Stolen=9680 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1334
> Query Plan=8627 Optimizer=0
> General=1816
> Utilities=9 Connection=280
> 2006-03-07 09:42:12.80 spid83 Global Memory Objects: Resource=1493
> Locks=214
> SQLCache=145 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:42:12.80 spid83 Query Memory Manager: Grants=0 Waiting=0
> Maximum=210349 Available=210349
> 2006-03-07 09:42:12.81 spid83 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:42:12.81 spid83 Insufficient memory available..
> 2006-03-07 09:43:01.86 spid83 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:43:01.88 spid83 Buffer Distribution: Stolen=1267 Free=6013
> Procedures=8415
> Inram=0 Dirty=10511 Kept=0
> I/O=0, Latched=126, Other=238969
> 2006-03-07 09:43:01.88 spid83 Buffer Counts: Commited=265301
> Target=283566 Hashed=249606
> InternalReservation=304 ExternalReservation=0 Min Free=128
> 2006-03-07 09:43:01.88 spid83 Procedure Cache: TotalProcs=2490
> TotalPages=8415 InUsePages=4895
> 2006-03-07 09:43:01.88 spid83 Dynamic Memory Manager: Stolen=9682 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1334
> Query Plan=8635 Optimizer=0
> General=1816
> Utilities=9 Connection=280
> 2006-03-07 09:43:01.88 spid83 Global Memory Objects: Resource=1493
> Locks=186
> SQLCache=145 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:43:01.88 spid83 Query Memory Manager: Grants=0 Waiting=0
> Maximum=208224 Available=208224
> 2006-03-07 09:43:01.88 spid83 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:43:01.88 spid83 Insufficient memory available..
> 2006-03-07 09:44:51.07 spid83 WARNING: Failed to reserve contiguous
> memory of Size= 393216.
> 2006-03-07 09:44:51.07 spid83 Buffer Distribution: Stolen=1246 Free=5938
> Procedures=8451
> Inram=0 Dirty=10539 Kept=0
> I/O=0, Latched=125, Other=239070
> 2006-03-07 09:44:51.07 spid83 Buffer Counts: Commited=265369
> Target=276731 Hashed=249734
> InternalReservation=304 ExternalReservation=0 Min Free=128
> 2006-03-07 09:44:51.07 spid83 Procedure Cache: TotalProcs=2519
> TotalPages=8451 InUsePages=4915
> 2006-03-07 09:44:51.07 spid83 Dynamic Memory Manager: Stolen=9697 OS
> Reserved=1392
> OS Committed=1355
> OS In Use=1334
> Query Plan=8669 Optimizer=0
> General=1819
> Utilities=9 Connection=284
> 2006-03-07 09:44:51.07 spid83 Global Memory Objects: Resource=1493
> Locks=145
> SQLCache=146 Replication=2
> LockBytes=2 ServerGlobal=43
> Xact=29
> 2006-03-07 09:44:51.07 spid83 Query Memory Manager: Grants=0 Waiting=0
> Maximum=203051 Available=203051
> 2006-03-07 09:44:51.08 spid83 Error: 17803, Severity: 20, State: 12
> 2006-03-07 09:44:51.08 spid83 Insufficient memory available..
> 2006-03-07 09:48:35.73 spid53 WARNING: Failed to reserve contiguous
> memory of Size= 2883584.
> 2006-03-07 09:48:35.81 spid53 Buffer Distribution: Stolen=1852 Free=254
> Procedures=8279
> Inram=0 Dirty=8797 Kept=0
> I/O=0, Latched=123, Other=275965
> 2006-03-07 09:48:35.81 spid53 Buffer Counts: Commited=295270
> Target=295270 Hashed=284885
> InternalReservation=301 ExternalReservation=60 Min Free=280
> 2006-03-07 09:48:35.81 spid53 Procedure Cache: TotalProcs=2532
> TotalPages=8279 InUsePages=4576
> 2006-03-07 09:48:35.81 spid53 Dynamic Memory Manager: Stolen=10113 OS
> Reserved=1376
|||There are at least a couple of situations that lead to this
error that are fixed in SP 4. You may also want to check the
network packet size being used by the clients. You can
capture this in profiler. See the following for more
information:
http://support.microsoft.com/?id=903002
Some processes that can affect the mem to leave space would
be sp_OA* procedures, most extended stored procedures, XML
operations.
-Sue
On Fri, 17 Mar 2006 02:23:57 -0800, David
<David@.discussions.microsoft.com> wrote:
[vbcol=seagreen]
>Please could I have a response to this post - it has now been over 3 business
>days.
>I'm looking for help with where I should start my investigation - I don't
>understand the reason for the 'not enough storage space' message, as there is
>plenty of storage space! The error logs suggest a problem with memory -
>'Failed to reserve contiguous memory of Size= 262144'. Presumably this is the
>underlying problem? If so, it must be due to something that has changed on
>all the servers, as we have now seen the problem on 2 of the 4 instances (1
>from each cluster). Should I be looking at the jobs that are running with a
>view to identifying memory leaks, or is it more likely to be due to something
>that has changed at OS level - perhaps a patch that has been applied?
>Finally, does SQL Server SP4 contain anything to specifically fix this
>problem?
>At the moment, we're getting round the issue by stopping and restarting
>instances every night, which is obviously far from satisfactory.
>Regards,
>David.
>"David" wrote: