I was using Drupal 6.2, TinyMCE3.0.7 and IMCE 6.1.x.1. I am sure that before long these hassles will be tied up by their very capable supervisors, but my notes might just help those who somehow are stuck with old code.

It only worked by assembling two separate bits of code patches. There was a comment on the Drupal main site from UFKU the IMCE author. He tells us to add this fragment to your template.php

function phptemplate_tinymce_theme($init, $textarea_name, $theme_name, 
                 $is_running) {
  static $access, $integrated;

  if (!isset($access)) {
    $access = function_exists('imce_access') && imce_access();
  }

  $init = theme_tinymce_theme($init, $textarea_name, $theme_name, $is_running);

  if ($init && $access) {
    $init['file_browser_callback'] = 'imceImageBrowser';
    if (!isset($integrated)) {
      $integrated = TRUE;
      drupal_add_js("
function imceImageBrowser(field_name, url, type, win) {
  tinyOpenerWin = win, tinyTargetField = field_name;
  if (typeof tinyImceWin == 'undefined' || tinyImceWin.closed) {
    tinyImceWin = window.open(Drupal.settings.basePath +'?q=imce', '', 
                        'width=760,height=560,resizable=1');
    tinyImceWin['imceOnLoad'] = function () {
      tinyImceWin.imce.highlight(url.substr(url.lastIndexOf('/')+1));
      tinyImceWin.imce.setSendTo(Drupal.t('Send to @app', {'@app': 'TinyMCE'}),
                        function(file) {
        window.focus();
        tinyOpenerWin.focus();
        $('#width', tinyOpenerWin.document).val(file.width);
        $('#height', tinyOpenerWin.document).val(file.height);
        $('#'+ tinyTargetField, tinyOpenerWin.document).val(file.url).focus();
      });
    }
  }
  else {
    tinyImceWin.imce.highlight(url.substr(url.lastIndexOf('/')+1));
  }
  tinyImceWin.focus();
}
", 'inline');
    }
  }

  return $init;
}

Separately I found this fragment from advice by a colleague who was also using an early Drupal 6. Who knows where he got it from? Anyway, add it to the end of your modules/imce/imce.module.

function imce_footer() {
  if (user_access('access imce')) {
    $header = drupal_get_js();
    //set imce if tinymce is in action
    if (!imce_integrate('tinymce', 'check') && module_exists('tinymce') 
                && variable_get('imce_settings_tinymce', 1) 
                && strpos($header, 'tiny_mce')) {
      $output .= '<script type="text/javascript" src="'. base_path() 
                  . drupal_get_path('module', 'imce') .'/imce_set_tinymce.js' 
                  .'"></script>';
    }
    //set imce if fckeditor is in action
    if (!imce_integrate('fck', 'check') && module_exists('fckeditor') 
                && variable_get('imce_settings_fck', 0) 
                && strpos($header, 'fckeditor.js')) {
      $output .= '<script type="text/javascript" src="'. base_path() 
                 . drupal_get_path('module', 'imce') .'/imce_set_fck.js' 
                 .'"></script>';
    }
    if ($output) {
      return '<script type="text/javascript"> var imceBrowserURL = "'
         .url('imce/browse').'"; </script>'. $output;
    }
  }
}
/**
 * integrate with editors.
 */
function imce_integrate($e, $check=NULL) {
  static $state = array();
  if ($check) {
    return $state[$e];
  }
  if (!$state[$e]) {
    drupal_add_js(drupal_get_path('module', 'imce').'/imce_set_'.$e.'.js');
    drupal_add_js('var imceBrowserURL = "'.url('imce/browse').'";', 'inline');
    $state[$e] = TRUE;
  }
}

Please don't ask me to explain. If it works, count your blessings, tidy up and move on.