FAQs |  Fixes |  Projects |  Mods and Utilities

Fixes for SMF 1.1 versions

Simple Machines Forum version 1.1 has several serious bugs, that the developers have stated will never be fixed. Therefore I am presenting them here so that SMF owners can incorporate them into their version 1.1 sites and not be plagued by these bugs.

Last update: 06 May 2012

You will notice that I am not using the standard SMF "mod" or "patch" system. That system is very badly implemented and more trouble than it's worth. Easily half the requests for help on the simplemachines.org community involve mods that either won't install, didn't install correctly, or failed to cleanly uninstall. Therefore, I am asking the user to use a simple text editor to install the code for these changes.

Of course, you should make a backup of at least the files being modified, in case you make an editing error and the code doesn't work any more. At least, you'll have a (somewhat) working copy to fall back on.

NEITHER CATSKILL TECHNOLOGY SERVICES, LLC, NOR ITS OFFICERS, MANAGEMENT, OR EMPLOYEES, SHALL ACCEPT ANY LIABILITY FOR ANY LOSS OR DAMAGE OF ANY KIND RESULTING FROM THE USE OF THESE INSTRUCTIONS, OR OF YOUR INABILITY TO SUCCESSFULLY UNDERSTAND AND FOLLOW THESE INSTRUCTIONS. IT IS UP TO YOU, AS THE OWNER AND MAINTAINER OF YOUR FORUM, TO FULLY UNDERSTAND "WHAT" IS BEING DONE BY THESE CODE CHANGES, AS WELL AS "WHY", AND TO ACCEPT ANY RISKS INVOLVED. IN OTHER WORDS, IT IS UP TO YOU TO UNDERSTAND THE CODE CHANGES BEING MADE, AND TO ACCEPT THAT EARLIER OR LATER VERSIONS OF SMF (SIMPLE MACHINES FORUM), OR CURRENT VERSIONS THAT HAVE BEEN MODIFIED IN SOME WAY, MAY OR MAY NOT WORK CORRECTLY WITH THESE CHANGES MADE. I WILL ATTEMPT TO NOTE HERE WHEN AND IF IT IS DISCOVERED THAT THESE CHANGES DO NOT WORK WITH SOME LEVEL (VERSION) OF SMF, BUT I CAN PROMISE OR GUARANTEE NOTHING. REMEMBER TO MAKE A BACKUP OF MODIFIED FILES, SO THAT YOU CAN EASILY UNDO YOUR CHANGES, AND TEST YOUR CHANGES IN WHATEVER WAY YOU CAN.

In the following instructions, the root directory "/" is intended to be the forum's base install directory. E.g., if you have your forum in http://www.domain.com/forum/, you would change "/" to "/forum/" in the instructions.

Comments and corrections should be PM'd to MrPhil on the simplemachines.org community forum. You are granted permission to link to this page, or any point on it, so long as you do not obscure or change any part of it, or claim authorship.

 

Settings.php file being constantly corrupted

Reference: remove last error code from Settings.php. Also see issues 4092 and 4222.

The design of SMF's mechanism for logging database errors is unbelievably bad. What happens is that the entire Settings.php file is completely rewritten just to update the timestamp of the last database error, which is assigned to PHP variable $db_last_error. The original file's settings are read in, then the file is emptied (truncated)! This is supposedly done to get around a glitch in some ancient version of a Unix-based server. Finally, the Settings.php file is written out anew, with all the original settings, except that $db_last_error has been updated with the current timestamp.

The problem with this approach is that there is a window in time where the original Settings.php file has been emptied, but the file not yet rewritten. This sets up a race condition, where if another user has also encountered a database error, their error code will read in the empty Settings.php file, and try to process that! What are the odds of this happening? Apparently, quite good. If one user encounters a database error, how likely is it that the problem will resolve itself before the next user attempts a database operation, and also gets an error? Not so good, from the number of desperate cries for help that pour into the SMF community forum, as well as hosting support forums. The symptoms of an emptied out Settings.php file vary, including error messages that "$sourcedir" is missing, as well as errors opening files such as /QueryString.php (most of the path, supplied by $sourcedir, is missing). The different ways that this error can manifest itself is confusing even to experienced support team members, is undocumented, and thoroughly crashes a forum. It is a catastrophe for its victims, but the normal response is to tell them to find a better host, one who never has database errors.

What's the solution? One simple solution is to make the Settings.php file "read only". On a Linux server, this involves setting its permissions to 444 or lower (644 if PHP runs as "group" or "other" — the idea is that the application can't write to the file). The drawback to this is that the database error timestamp can't be logged, and the user may receive an error log entry reporting that the file was unwritable. A second solution would be to fix the code in the Settings.php file update function that appears to be a check for an empty Settings.php file, and to either forget about updating the file, or to wait until it can read in a good version. Evidently that code is broken.

My solution is to separate out the database timestamp line into its own file. Only that file, with its single line of code $db_last_error = 1234567890;, gets rewritten. If it gets corrupted (emptied out) by the race condition, it's only a minor problem. There is a check in Settings.php to see if $db_last_error was defined, after it was supposedly included into the file. If not, assume that it was empty and rewrite it with a 0 value. That may not be the optimal solution, as another user's SMF process may be busy writing out the revised file (with a non-zero value), but at least it will prevent all these terrible forum-crashing problems.

Code Changes

In /Settings.php, approximately line 55, find

$db_last_error = 0;

Replace it with

// timestamp of last database error detected include($boarddir . '/LastError.php'); // despite our best efforts, was this file emptied out? if (!isset($db_last_error)) { updateLastError(0); // just give it a 0 value // might also/instead try to repair DB here $db_last_error = 0; }

Create a new file named /LastError.php. It should be in the same directory (folder) as Settings.php.

<?php $db_last_error = 0; ?>

Add a new function to /Sources/admin.php to update just the LastError.php file, rather than the entire Settings.php file. This new code is inserted after the end of the updateSettingsFile() function (closing }).

// Avoid calling updateSettingsFile for $db_last_error. If there's one error, // there's often multiple errors, and two updateSettingsFile() calls too close // together can be disastrous, especially if a second one is invoked just // after Settings.php is truncated (zeroed out) and before the new copy is // written. There's no guarantee that LastError.php won't get similarly hosed, // but at least it can be easily automatically fixed. function updateLastError($time) { global $boarddir; // Blank out the file - done to fix an oddity with some servers. $fp = @fopen($boarddir . '/LastError.php', 'w'); // Is it even writable, though? if ($fp) { fclose ($fp); $fp = fopen($boarddir . '/LastError.php', 'r+'); fwrite($fp, "<?php\n"); fwrite($fp, '$db_last_error = ' . $time . ";\n"); fwrite($fp, '?' . ">\n"); // to prevent premature closing of PHP code fclose($fp); } }

In /Sources/Errors.php, at approximately line 195, change the code to not rewrite the whole Settings.php file, but just the LastError.php file. Find

updateSettingsFile(array('db_last_error' => time()));

and replace it with

updateLastError(time());

Do the same thing in /Sources/Subs-Auth.php, at approximately line 364. Find

updateSettingsFile(array('db_last_error' => time()));

and replace it with

updateLastError(time());

Go to top

 

Bad database dumps (exports) to .sql file

There are a couple of serious bugs in the code that exports (dumps) the database to an .sql file.

Superfluous INSERT

If a table has an exact multiple of 250 rows, there will be a superfluous INSERT INTO command generated at the end of the table, but with no data (no VALUES clauses). This will generate an SQL syntax error upon any attempt to import the .sql file. The error isn't hard to fix (remove a few extra lines in .sql file), but if the .sql file is very large or is compressed, it can be a hassle to deal with.

One of the functions in the Sources/DumpDatabase.php file was extensively restructured to ensure that there was at least one more row of data to be output, before issuing an INSERT INTO command.

Bad numeric values in hex strings

Any value (destined for a VALUES clause) that looks like a number doesn't get single quotes around it. The PHP is_numeric() call is used, but it seems to have a bug of its own. It apparently will pass what appear to be floating point (real, e-notation) numbers, but these numbers can not be represented in hardware. I have seen this with hexadecimal strings that happen to be all decimal digits (0 to 9), with a single 'e' somewhere in the middle, e.g., 123e4567. This appears to pass the is_numeric() test, but when MySQL attempts to read in the data, it will choke on it with some sort of error.

The same function as before was simply modified to put single quotes around all values, no matter whether they appeared to be numbers or not.

Bad default value for numbers in .sql file

When SMF writes out the field definitions for a table in the database dump (export), sometimes an integer value can be given a default value of ''. This is not correct — it must be '0'.

Not yet coded

Reserved keywords as column names

When SMF writes out the field definitions for a table in the database dump (export), it may leave column (field) names bare. This is not a problem if the names are not SQL reserved keywords (e.g., DESC), but will cause problems if they are. The easiest fix is to wrap the name in backticks `. Strictly speaking, this is not an SMF backup bug, but an enhancement to keep SMF and mod design errors from causing bad backups.

Not yet coded

Code Changes

In /Sources/DumpDatabase.php, starting at approximately line 45, and continuing through approximately line 51, find the following code:

string getTableSQLData(string table_name) - dumps the CREATE for the specified table. (by table_name.) - returns the CREATE statement. */ // Dumps the database to a file. function DumpDatabase2()

Insert 6 lines (in bold here) into the middle:

string getTableSQLData(string table_name) - dumps the CREATE for the specified table. (by table_name.) - returns the CREATE statement. */ /* How many rows to include in one INSERT statement. Default is 250 for speed, but you may wish to take it down to 10 or 20 for debugging purposes (to show the entire INSERT INTO command, rather than just the beginning). */ $rowsPerInsert = 250; // Dumps the database to a file. function DumpDatabase2()

In /Sources/DumpDatabase.php, starting (originally) at approximately line 196, and continuing through approximately line 254, find the following single function:

// Get the content (INSERTs) for a table. function getTableContent($tableName) { global $crlf; // Get everything from the table. $result = db_query(" SELECT /*!40001 SQL_NO_CACHE */ * FROM `$tableName`", false, false); // The number of rows, just for record keeping and breaking INSERTs up. $num_rows = @mysql_num_rows($result); $current_row = 0; if ($num_rows == 0) return ''; $fields = array_keys(mysql_fetch_assoc($result)); mysql_data_seek($result, 0); // Start it off with the basic INSERT INTO. $data = 'INSERT INTO `' . $tableName . '`' . $crlf . "\t(`" . implode('`, `', $fields) . '`)' . $crlf . 'VALUES '; // Loop through each row. while ($row = mysql_fetch_row($result)) { $current_row++; // Get the fields in this row... $field_list = array(); for ($j = 0; $j < mysql_num_fields($result); $j++) { // Try to figure out the type of each field. (NULL, number, or 'string'.) if (!isset($row[$j])) $field_list[] = 'NULL'; elseif (is_numeric($row[$j])) $field_list[] = $row[$j]; else $field_list[] = "'" . mysql_escape_string($row[$j]) . "'"; } // 'Insert' the data. $data .= '(' . implode(', ', $field_list) . ')'; // Start a new INSERT statement after every 250.... if ($current_row > 249 && $current_row % 250 == 0) $data .= ';' . $crlf . 'INSERT INTO `' . $tableName . '`' . $crlf . "\t(`" . implode('`, `', $fields) . '`)' . $crlf . 'VALUES '; // All done! elseif ($current_row == $num_rows) $data .= ';' . $crlf; // Otherwise, go to the next line. else $data .= ',' . $crlf . "\t"; } mysql_free_result($result); // Return an empty string if there were no rows. return $num_rows == 0 ? '' : $data; }

Replace it with the following function:

// Get the content (INSERTs) for a table. function getTableContent($tableName) { global $crlf, $rowsPerInsert; // Get everything from the table. // note: this might have been done in a loop with LIMIT n, 250 $result = db_query(" SELECT /*!40001 SQL_NO_CACHE */ * FROM `$tableName`", false, false); // The number of rows, just for record keeping and breaking up INSERTs. $num_rows = @mysql_num_rows($result); $num_fields = mysql_num_fields($result); $current_row = 0; if ($num_rows == 0) return ''; $fields = array_keys(mysql_fetch_assoc($result)); mysql_data_seek($result, 0); // build up string to return $data = ''; // Loop through each row. There is at least one. while ($row = mysql_fetch_row($result)) { // $row needs to be output, and current_row is still the // number of the previous row output (starting with 0) // every 250 rows (current_row = 0, 250, 500,...) output INSERT INTO header if ($current_row % $rowsPerInsert == 0) { if ($current_row > 0) $data .= ';' . $crlf . "\t"; // end previous INSERT $data .= 'INSERT INTO `' . $tableName . '`' . $crlf . "\t(`" . implode('`, `', $fields) . '`)' . $crlf . 'VALUES '; } else { // in the middle of an INSERT, add comma to separate this value group from previous $data .= ',' . $crlf . "\t"; } $current_row++; // Get the fields in this row... $field_list = array(); for ($j = 0; $j < $num_fields; $j++) { // Try to figure out the type of each field. (NULL, number, or 'string'.) if (!isset($row[$j])) $field_list[] = 'NULL'; elseif (is_numeric($row[$j])) // used to put out bare number, even if the field is character! $field_list[] = "'" . $row[$j] . "'"; else $field_list[] = "'" . mysql_escape_string($row[$j]) . "'"; } // 'Insert' the data. Ends with bare ')', needs comma or ; $data .= '(' . implode(', ', $field_list) . ')'; } // close off the open INSERT, must have at least one value group $data .= ';' . $crlf; mysql_free_result($result); // Return an empty string if there were no rows. return $num_rows == 0 ? '' : $data; }

For your convenience, the above code segment is provided as a file: getTableContent.code. Warning: this is just one part of the DumpDatabase.php file — it is not a replacement for the whole thing!

Go to top

 

Fix to password change

Ref: this posting, SMF has problems in changing your password when the old one includes certain odd characters. The code change, copied here, is in Sources/Profile.php. Find

// Bad password!!! if (!$good_password && $user_info['passwd'] != sha1(strtolower($user_profile[$memID]['memberName']) . $_POST['oldpasswrd'])) $post_errors[] = 'bad_password';

and change it to

// Bad password!!! if (!$good_password && $user_info['passwd'] != sha1(strtolower($user_profile[$memID]['memberName']) . un_htmlspecialchars(stripslashes($_POST['oldpasswrd'])))) $post_errors[] = 'bad_password';

Go to top

 

Database Upgrade Required

Some error messages in SMF will inform you if your SMF database version doesn't match the current SMF code version. As the database format changes very slowly, this warning is unnecessary and merely causes panic among site owners. The last time the database layout (version) changed was with SMF 1.1.9, so up through (at least) SMF 1.1.11, there's no need to "upgrade". The upgrade should be done automatically (run upgrade.php) when SMF is upgraded, but a fair number of site owners seem to miss that step.

If your SMF database is genuinely out of date (e.g., is pre-1.1.9), you can get a copy of upgrade.php, upgrade_1-0.sql, and upgrade_1-1.sql from the large upgrade package, copy them into your forum's root (where Settings.php lives), and run upgrade.php from your browser. Once the upgrade is done, erase those three files.

If your SMF database is at 1.1.9 or higher, and you just want to get rid of those annoying "upgrade required" messages, go into phpMyAdmin, go to your SMF database, go to the settings table (e.g, "smf_settings"), and change the "smfVersion" field to "1.1.11" (or whatever your current version is). This should shut up the annoying warning.

SMF can be patched in the code that checks the database level, and suppress the warning if it's unnecessary. Find the following code in Sources/Errors.php (SMF version 1.x) or Sources/Subs-Db-mysql.php (SMF version 2.x) (We don't know if -sqlite and -postgresql versions use the same numbering system):

// A database error is often the sign of a database in need of updgrade. Check forum versions, and if not identical suggest an upgrade... (not for Demo/CVS versions!) if (allowedTo('admin_forum') && !empty($forum_version) && $forum_version != 'SMF ' . @$modSettings['smfVersion'] && strpos($forum_version, 'Demo') === false && strpos($forum_version, 'CVS') === false) $context['error_message'] .= '<br /><br />' . $txt['database_error_versions']; if (allowedTo('admin_forum') && isset($db_show_debug) && $db_show_debug === true) { $context['error_message'] .= '<br /><br />' . nl2br($db_string); } // It's already been logged... don't log it again. fatal_error($context['error_message'], false); }

Change this section to: (NOTE that this code is for 1.1 and may need modification for 1.0 or 2.0)

// A database error is often the sign of a database in need of upgrade. Check forum versions, and if not identical suggest an upgrade... (not for Demo/CVS versions!) if (allowedTo('admin_forum') && !empty($forum_version) && $forum_version != 'SMF ' . @$modSettings['smfVersion'] && strpos($forum_version, 'Demo') === false && strpos($forum_version, 'CVS') === false) { // innermost arrays are starting and ending database // versions, and corresponding forum versions. // UPDATE with new SMF releases! $compatibleVersions = array( '1.0' => array( array( '0', '16', '0', '16' ), array( '17', '19', '17', '19' ), array( '20', '22', '20', '22' ), ), '1.1' => array( array( '0', '8', '0', '8' ), array( '9', '11', '9', '11' ), array( '12', '16', '12', '16' ), ), '2.0' => array( array( '0', '2', '0', '2' ), ), ); // suppress warning if same database layout $DBVer = explode('.', $modSettings['smfVersion']); // e.g., '1', '1', '9' if (count($DBVer) == 2) $DBVer[2] = '0'; $ForumVer = explode('.', substr($forum_version, 4)); // e.g., '1', '1', '11' after dropping 'SMF ' if (count($ForumVer) == 2) $ForumVer[2] = '0'; $OK = false; $majorMinor = $DBVer[0] . '.' . $DBVer[1]; if (isset($compatibleVersions[$majorMinor]) && $DBVer[0] == $ForumVer[0] && $DBVer[1] == $ForumVer[1]) { // loop through valid ranges for ($i=0; $i<count($compatibleVersions[$majorMinor]); $i++) { // so long as DB and FV revision numbers fall within // this range, we're good if ($DBVer[2] >= $compatibleVersions[$majorMinor][$i][0] && $DBVer[2] <= $compatibleVersions[$majorMinor][$i][1] && $ForumVer[2] >= $compatibleVersions[$majorMinor][$i][2] && $ForumVer[2] <= $compatibleVersions[$majorMinor][$i][3]) { $OK = true; break; } } } if (!$OK) { $context['error_message'] .= '<br /><br />' . $txt['database_error_versions']; } } if (allowedTo('admin_forum') && isset($db_show_debug) && $db_show_debug === true) { $context['error_message'] .= '<br /><br />' . nl2br($db_string); } // It's already been logged... don't log it again. fatal_error($context['error_message'], false); }

There sort of seems to be a 1.0.20 upgrade, but it's not clear. Therefore, we are listing 1.0.20's database as incompatible with 1.0.19's, until the matter is clarified.

Note that 1.1.12 new installations change all tables from TYPE=MyISAM to ENGINE=MyISAM. As there is an "upgrade.php" to do this, just to be safe, we have declared 1.1.12 to be incompatible with 1.1.9-11. The SMF install.php routine changes ENGINE= back to TYPE= for MySQL v3.

Note that Version 2.0 "Beta" and "RC" (Release Candidate) levels are not covered here, only "Gold" 2.0 and up.

Go to top

© Copyright 2010–2011 by Catskill Technology Services, LLC