HP Proliant servers that use the P410 Smartarray controller use batteries that are supposed to last around 3 years, but on some hardware that’s getting on a bit now, they are a serious concern, as they still cost about £60 each or £120 if you buy them from HP. And everyone knows a P410 without Battery Backed Write Cache (BBWC) is seriously compromised!
So here is our cheapskates way around futureproofing our older Proliant DL385’s.
Prise open the old battery casing and toss out the old cells, wire in a new lead to some standard Ni-MH AAA cells, and your ready to go… OK, some pictures might help :)
Below then is the item were discussing – its HP part number is 381573-001 or 398648-001 or one of the other vast number of variants, they may have slightly different power ratings, the important one for us is 4.8V Ni-MH, that tells us there are 4 cells internally.
At first appearance the block is sealed and there is no obvious way to get to the cells, in fact its glued together and covered with a thin plastic sheet. Our first task is to force our way into the cell compartment and remove the plastic lid, this is where it gets destructive, don’t proceed unless you have no further use for the battery pack!
Prise a small screwdriver into the edge of the plastic, until you’ve made a hole and then trace around the edge to reveal the cells, but be careful not to go to far in or puncture the cells or insulation. Starting in the middle of the long sides is a good place as shown above.
The cells are held in with silicone gunk and double sided tape so its fairly easy to prise the lid off and the cells out of their home.
Take care with the cells as they could still have some power in them, even if the server disagrees! Lever them out until they are just attached with the leads to the small PCB.
The PCB simply unclips from the small cutout in the middle. Pull this clear and you can see the battery is just held against two gold plated pads. Now the cells can be removed from the case. The original cells we had were Varta V500HT 500mAh cells which are standard Ni-MH 1.2V cells attached 4 in series. We now purchased a small AAA battery holder for 4 cells and a PP3 style clip with flying leads (all from Maplin if you have one handy). A small 3mm hole was drilled in the side of the old cell holder to feed the wires in and through to where the PCB fits. Note in the picture below, the ‘+’ symbol adjacent to the red wire, this is important to get right! Simply solder the new battery leads to the gold plated pads.
The small PCB is inverted above which is why the wires are crossed. Now you can reassemble the small board in its holder again, and it should end up looking like the assembly below. Note again the red wire by the ‘+’ sign.
Finally reassemble into the server, the extra battery pack found a neat little hollow in our DL385, your solution may need to adapt to the different servers that these packs are used in. We put a small amount of Velcro under the battery pack just to keep it in place. The cells used were own brand Maplin NiMH cells rated at 1000mAh. Don’t go too crazy in cell capacity as the charging current is limited by the control board. Total cost is around £6 per server, if you’ve got a number of servers this is very welcome!
Restart the server and it should take 2-3 hours of recharging before your cache comes back online.
Don’t forget to dispose of the old NiMH cells properly, most supermarkets seem to collect them now.
The neat little utility hpacucli now gives us this status:
Post Prompt Timeout: 0 secs Cache Board Present: True Cache Status: OK Accelerator Ratio: 25% Read / 75% Write Drive Write Cache: Enabled Total Cache Size: 512 MB No-Battery Write Cache: Disabled Cache Backup Power Source: Batteries Battery/Capacitor Count: 1 Battery/Capacitor Status: OK
Thanks for the write up, I started taking one of these apart yesterday and was googling the battery type when I found this page. Your work confirmed my theory I could botch my own battery pack together :)
My DL380 G5 is happily charging it’s new friends. Total cost of repair was £3.24.
Thanks for this post. I recently built a home server with a P400 as the storage controller, and a few days after installing the controller the BBWC started giving a short circuit error (which would go for a few hours if I restarted the server but then come back).
Following your write up, I bought this battery pack on ebay and replaced the BBWC’s battery. After taking the original battery cells, I cut the sides of the battery compartment with a dremel, and after soldering the new battery leads glued the battery on the back of the BBWC battery compartment.
This worked like a charm! In my case, it was a DL360 G5. I purchased a 4xAAA case (in a single row). I inserted Toshiba Impulse 750ma batteries. You mentioned 1000ma but I couldn’t find any locally that weren’t generic Chinese batteries. For mounting, the server had a dummy optical drive insert, so I cut off the back half of it, attached “cushion tape” to the top and bottom of the battery holder, and slid it into the back of the optical drive bay. It’s not perfectly secured. Battery status is “fully charged” in hpacucli now :)
Just wanted to say thanks for this also. Had the option of $70 and 2 weeks delivery for a used BBWC versus $10 and 5 minutes of my time. Works great.
Great article, thank you! I’m going to try this in a DL365 which doesn’t have any spare space to stick an extra battery holder, so I found that four of the 2/3A size NIMH cells should almost fit in the original compartment. A pack of four of them will be slightly wider and longer than the compartment, so some hacking of the plastic will be needed to make it work. I will enlist the help of a local Batteries Plus who stocks such cells and has staff that can custom-assemble a pack.
(I wonder if we could just use the capacitor from the P410 though instead of going to all this trouble?)
I’m looking at something similar for my aging DL360/380 G5 servers. Those original P400 batteries are dying quickly now that they’re 5+ years old… I’ve got 4 dead ones now out of a dozen systems (and I recall replacing one or two already in the past couple years).
I wondered about the supercaps in the P410/P420 models, but those are meant for very short term use… just enough time for the flash backed cache to take all of the contents of the fast DRAM and write it out to the flash RAM.
The tech specs on the supercaps are 17 Farad at 4.8V (it’s actually a pair of 35 Farad 2.7 volt supercaps in series, according to their documentation… I haven’t pried one open myself). If I had to guess, 17 F charged to 4.8 V could probably output suitable current for a few minutes before the voltage drops below what the circuitry needs. But I don’t know what the current draw is for the FBWC module… could be a few milliamps, or a couple hundred milliamps. I’m sure it only takes a few seconds to write 1-2 GB of cache memory to some flash, so they probably built in a fudge factor of 2-3 times the write-time needed, but that’s about it.
The battery packs are meant to last 48 hours… which HP assumes should be enough time to either get your server up and running again so the P400 can write the memory contents out to disk, or even swap the cache module (with battery still attached) onto a replacement array controller, in case it was the controller itself that died.
My trouble is what others have noticed… the DL360/365 models have precious little space inside, especially if you actually do have a DVD drive installed. There’s an empty drive bay below where the battery normally goes, so I might cut up the drive blank so it’s not hogging all of that space, and leaves me enough to put in a 4-AAA side-by-side pack, and I’d probably need to make the wiring detachable so I could feed it through the little gap and “plug in” to the re-wired old battery pack.
By the way, in my poking around, I realized that the smart array controller doesn’t like to start up if you’ve unplugged the battery module from either end. You can have the batteries themselves removed from that little controller board on the battery module… it just treats the missing batteries like it would if they were bad or something. But you do have to be plugged into that little circuit board or the SA controller gives an error on POST.
I tried this on a 410(380 G6) . I used 4xAA batteries and initially the voltage ran a little high ~ 5.2 V so I let them discharge a little. I shut down the server and made sure to get the polarity right, powered it up and the pcb board just blinks amber-green (left it this way for a couple days). I tried another battery from another 410 (swapped out the battery so Im using the same pcb board) and it worked just fine. So Im not sure why this didnt work for me
Hmm, Were they NiMh rechargeable batteries? They should only be 1.2v nominal per cell, so anything over 4.8v sounds wrong. I’ve modded about a dozen systems like this and so have others, never had an issue!
This is great! I’ve just been researching specs on batteries after tearing one of these apart, and it was nice to find that this has been done before.
Very nice! I have a question about this mod…in a few years when the batteries fail again and need to be replaced, can they just be “hot swapped” from the holder, or will the server need to be shutdown? I’m planning on doing this after buying a replacement HP battery module that came in DOA and won’t charge. The manufacture date was actually a month OLDER than the one I replaced!
No idea, but for safety I would go with the shutdown once every few years! :)
Good hack :)
How does the high temperature inside the server affect the batteries’ stability ?
Hi, Well its just replacing one NiMH cell with another one, so depends on what cell’s you put in there.
However it’s not that hot in our servers and we’ve seen no issues with the replacement cells.
I tried the supercaps (2x 2,7V 10F) but no joy, battery’s logic board blinking amber led. AAA-s worked fine.
Just swapped one, and will replace another later today
Maplin bits:
JG79L – battery box
L85AB – PP3 clip
L29BJ – lots of AAA batteries
Got this in a non HP server, so the battery box sits nicely in the space where the old ones were removed – even stuck there by itself as there’s lots of left over sticky in there.
A couple of minutes to pull apart and solder, job done!
Thanks!
Quick update – DIY batteries have been in a year and haven’t missed a beat! Work as well as the day they were installed.
Thanks!
Works like a charm. I slightly modified the wiring. Added an 2 pin connector between bbwc battery and rechargables case to be able change the AA to AAA without soldering again.
And didn’t solder wires to board, but to old contacts from dismantled cells instead, so board still could be detached. With no purpose at all. :)
My two cents: I did this mod to my old server, however I used a batterypack with soldered connections. I want it soldered all the way, no battery holder or connectors that can corrode and glitch. You do not want to risk any bad contact to these batteries or it may corrupt your data. And some kind of box to have them in is important so if they leak they wont destroy anything. Also only do this mod on servers out of warranty or HP will never help you. Someone was commenting on short battery life while in high temperatures inside the server, well if it gets that hot in your server you have other problems to worry about, it should never be a problem. ;-)
Thanks for the info – I found the original cells are sold by CPC/Farnell for around £13 so I managed to replace the original cells which does make a neater solution (although not as easy to replace when they die again clearly). I have a brief write up here: http://www.wickensonline.co.uk/rc2012sc/2015/02/01/raid-backup-battery-replacement/
Great post, just replaced the batteries on my server. A few tips, I found scoring the edges of the cover by running a sharp utility knife along the long edges helps when removing the cover, resulting in less warping of the case. I also just ran the battery wires through the small gap where the original battery cell contacts went through to the circuit board, and looped the extra length of wire in the empty space where the old cells used to live since I didn’t want to bother with any drilling. Then I used double sided tape to attach the AAA battery holder to the back of the BBWC battery unit.
A side note, I noticed the photos in the post show a 398648-001 battery, which appears to be used in the P400 and P800 Smart Array as opposed to the P410 mentioned in the text. According to HP, these BBWC units actually have 5000mAh, so running four 1000mAh batteries actually give 25% less capacity than OE. Of course, at around 5-10% of the cost, I’m sure no one is complaining.
The 462976-001 unit I pulled from my ML150 G6 server officially only has 650mAh, so after swapping out the old cells with 4 AAA GP ReCyko cells, I’m running at ~3600mAh (spec’d at 800mAh, but measured to ~900mAh per cell). HP claims that BBWC can hold data for at least two days, so by my calculations, the upgrade from 650mAh to ~3600mAh should give me at least 11 days… not that I’d ever hope to push that envelope.
Okay, after some thinking, 650mAh is unbelievably low for a battery unit with four cells, so I took a look at my old cells and what do you know? The 650mAh spec is per cell, so the entire battery unit would be 2600mAh. This means the new AAA batteries should give me around 38% extra capacity, not the ridiculous claim of 11 days I had earlier.
Hi, The number of cells won’t affect the capacity, 4x650mAh cells in series is still just 650mAh, just at 4.8V rather that 1.2V. I don’t think the capacity is critical, if you are relying on the BBWC to cover you for long periods, you need to look for better solutions, in most cases its just to keep the disk clean in the event of an unexpected reboot or similar situation, in a clean shutdown all dirty writes are flushed, the BBWC just holds any that can’t be written for whatever reason.
Hah, I feel a bit silly now, I’m forgetting all my high school science! I was just blabbing on, but I’m guessing that HP’s spec for the 398648-001 unit may be a typo then; perhaps they meant 500mAh (which matches your cell spec) instead of 5000mAh.
Just curious, what would the life expectancy of the AAA cells be? OEM BBWC units are ~3 years, would it be similar for our off-the-shelf AAA cells?
I would expect them to be the same as the OEM units, apart from the packaging they are just standard NiMH cells, so if the replacement AAA cells are decent quality they should be the same. Of course not all cells are good quality!
My P400 BBWC just died and did this. Worked sweet. It’s a bit hard to get the battery packs for these things here in NZ.
Thank you very much, I had previously bought several spare battery packs and most of which arrived damaged. This is a genius and effective way to stick it to the man! you are a godsend.
Great solution. I am in Alaska. On top of everything else, you can not buy a replacement locally. One small thing, I used a glue gun to mount the battery holder to the case.
Thanks,
Your research saved my day and home budget of some major expense.
Did the trick with 1000mah on P800 works great.
Hello,
Thanks for a great post. I did the same to replace a battery pack on my DL385 P400 controller. I employed a slight variation though, I first removed the old batteries as you did and then I cut off the plastic section that had contained the old batteries. Next I soldered one of these to the small metal plates on the PCB:
http://uk.rs-online.com/web/p/coin-button-rechargeable-batteries/7020755/
It worked out approx £17 including VAT and also because of the width of the battery pack it fitted perfectly into the space where you have put your battery holder. It fits so snugly it is almost as if it was made to go there. :) [Obviously there won’t be the same space on all server models]. It charged up within 2 hours and now my cache is back on line and working just fine.
Thanks again for the post and for saving me a load of money.
Worked Perfectly. I just soldered the batteries together after taping them together in a cube. Keeping the original leads to the board made it a bit simpler. Thanks!
Hello,
great tips and I like to do them myself for all my 3 servers.
but:
1) did anyone actually test the scenario of a power failure (using Raid5,Raid6) when using this battery replacement? I’m just wondering whether the characteristics of normal NiMh batteries might inhibit to successfully write back the cache to the disks.
2) Batteries: should I take normal NiMh batteries or low self-discharge (aka eneloop, Ready2Use) ones?
thanks!
Hi Tim,
The original cells are just standard NiMh cells in a compact casing, normal NiMh AA cells are a fine replacement, you dont need anything more fancy, the performance should be exactly the same, if not better!
The battery preserves the data in the otherwise volatile write buffer and then the controller writes the data to disk upon next power on so there is never any substantial load on the batteries.
I suspect any AAA Ni-Mh batteries will work, but probably a good idea to stay in the 650mah ballpark to avoid stressing the charge circuit.
I am using eneloop AAA 5th gen Ni-Mh (750mah) low self discharge batteries.
The controller reported write cache temporarily disabled and battery recharging from new, but status changed to OK / OK after about an hour.
If the current is really negligible then a low self discharge battery may preserve the data for longer, however this would only come in to play after months of no power and is probably irrelevant.
Work perfectly for me. Thanks so much for your guide. Really save the trouble for me since I can’t wait for the parts to arrives.
This is working great! Thank you for the article!
Thank You, You saved a lot $$$ for me too.
I’ve used 4 x AAA 1600 mAh rechargable batteries.
I think it at least doubled backup time for RAID memory cache as original is 4.8V 650 mAh. Used hot glue and transparent tape for mount.
Of course, I hope they won’t explode or brick memory module :) :) :)
Will see..
Keep doing good work!
Hmm, I used 4x AAA NiMH @ 900 mAh, measured voltage is ~4.8 V, but I get this error message, this message was also before I changed to the new one:
“POST Error: 1748-Slot X Drive Array – Unsupported Array Accelerator Battery Attached – Please install battery pack(s) with the correct part number.”
I’m unhappy, the old batteries was damaged and with the new it doesn’t work, is the small controler defect or what or where is the issue?
Hi Axel, If you had this message before I don’t think changing the battery itself will help, only the controller/charger board can talk to the BIOS, the battery is dumb, so I think you have a bigger problem.
Do you think the controller or charger board is damaged and doesn’t recognize new batteries? The old batteries were bloated (pregnant).
Hi Axel, If its the original controller and it used to work OK, then yes its damaged or mis-configured. If not it could be the wrong part?
I just wanted to say thank you for this blog post! I recently purchased a P410 card from eBay and as you can imagine, the batteries very quickly died on me due to old age. Looking for replacements online only turned up $100+ refurbished BBU’s, which was way more than I was willing to pay. Your solution was quick, simple and has the added advantage of allowing the batteries to now be replaced easily in the future. Total cost; around $4 and 15 minutes of my time. Thanks again!
thanks a lot for sharing this.
i was having a bad time finding batteries that actually works nowadays, i did not find a way to buy them from hp anymore.
And most of the one i got from ebay or amazone were just old crap (somtime older than the one i was trying to replace)
this tips is becomming even more valuable as time pass …
followed your tip, and it just took me 5 mnts to reborn a couple of old proliant DL580 G5 we still have running …
Just wanted to +1 this thread, great idea, thanks for sharing :))
Last night I replaced a failed BBWC battery in a DL360 G5 with this solution. I went a bit further than the OP and completely did away with the plastic carriage that the original unit came in. Installed a low profile 4x AA battery holder with PP3 battery clip (will list parts below) and 4x Amazon Basics AA rechargeable cells. I soldered the battery clip to the original PCB, and then hot glued it to the battery holder and insulated both the cells and the PCB in two separate sections of heat shrink plastic.
The batteries fit into the original location, and the PCB slid down between the disk backplane and the fans. The System Mgmt Homepage reported the batteries were charged and the BBWC active after about 2.5 hours.
All in all a very neat solution and the cost was < £10!
Parts list:
1x HF94 4AA Battery Box (Maplin)
1x HF28 PP3 clip (Maplin)
4x AmazonBasics AA 2000mAh cells – https://www.amazon.co.uk/dp/B00CWNMV4G
Thank you so much for this blog post!
Five years later and it still saves sysadmins day :-)
Hi!
Did this on 2 HP dl 360 g5 and i works great.
Now I did the same on a DL160 G6 but get an error that I have a unsupported batterypack. Does any one know what could be the problem?
Br,
Cribbe
Great article! I just did this for an old DL360 G5 w/ P400i and it works great. I now have RAID6 ADG support w/ caching enabled. There’s no other way to do this anymore since the only battery packs you can get are used from 3rd party vendors and already have dead cells.
I have the same issue the other 360 users on here ran into with space inside the chassis so I wound up taking out the DVD drive for now to accommodate the battery holder. I’m going to try George’s solution to get everything back into the battery compartment where it belongs. Like George, I used AmazonBasics batteries, but went with the AAA 750 mAh cells. I didn’t have the courage to try larger ones.
I’m also going to have to do this on a DL380 G5 soon as its battery pack is about to crack from the swelling. Hopefully I’ll find a little more space for the battery holder in that chassis.
Just repaired my batterypack, working great. Didn’t have to solder the wires to the pcb, just ripped the old copper connectors from the dead battery, peeled them into the blade of a small male-connector, and glued this connector to the bottom of the pack, thus fixing the copper connectors. Right now hpssacli tells me a “happy recharging”-;)
I arrived at almost this solution. You can’t get the 650mA cells but you can get exactly the same physical size 600mA cells from RSonline. I salvage the brass connectors from the dead cells as well as the clear plastic retainer that keeps them in place. As one of your respondents says, a sharp knife (Stanley with new blade) will free the top leaving it re-usable. The cells are all mounted the same way up, which I abandoned in favour of having two pairs, one set positive down nearest the charger board and the other positive up. Wires make it hard to fit the case lid back on. So I used copper ‘slug’ tape from the garden centre. One piece stuck under the top joins the two pairs together. A piece under the pair near the charger has the positive terminal soldered to it. A piece under the other pair has a thin ~3mm strip leading to the negative terminal. The thin strip is insulated with plastic tape to prevent it shorting the positive . Bend the tags on the cells up slightly, tape two cells together around their rims. When you put it all together using super glue or a glue gun, test the voltage across the brass connectors before re-inserting the charging board. Off load this will be about 5.03V. This reconstruction will fit back in the server like the original.
So awesome! Thank you! Worked perfectly the first time. Hardest part was soldering the tiny wires on the small board.
This totally works! Thank you so much for saving me so much money. The Battery/Capacitor Status shows OK after approximately 1-2 hours of “Recharging”.
I might come back in 6-12 months to post if any issues after running it like this for a longer period of time.
Hi wrenhill!
Awesome mode! I am going to attempt to do this on my DL160 G6 today, and I’m wondering what the max recommended mAh should be?
I’ve found these: https://www.kjell.com/no/produkter/elektro-og-verktoy/batterier/oppladbare-batterier/nimh-batterier/standardceller/kjell-company-oppladbare-aaa-batterier-1000-mah-4-pk–p42503 1.2v 1000mAh cells. Would that be fine? Also thinking of using these: https://www.kjell.com/no/produkter/elektro-og-verktoy/batterier/oppladbare-batterier/nimh-batterier/industriceller/gp-nimh-batteri-1-2-aa-1-2-v-750-mah-p32115 They are 1.2v 750mah and they fit inside the old enclosure.(Dl160 is tight on space)
Thanks in advance!
Hi, I think those will be fine, the size/rating is not critical, 400mAh – 1200mAh have all been used OK, obviously trading charging/dischare times :)
Hi!
Thank you for the great post. I succeeded in one case to do it on my HP DL360 G7.
The only working case was: 4 * AA 1.2V 2100mAh batteries. I tried with 4 * AAA 1.2V 800 and 1000mAH batteries but it did not worked. By not working it means that the amber led is blinking and the controller says to me to replace the current plugged battery.
All batteries were just newly bought. Any ideas why?
Thank you in advance.
Lo realize para HP DL360 G7
Utilize 4 baterias Energizer Recharge 1.2V 2000mAh
y funciona perfecto!!
A perfect mod! Does anyone have a description of how to install the hpacucli utillity on af Debian 10 (Buster) system on a Proliant ML350 Gen 6 machine ?